cachecpu-cache
While I understand that the computer cache is:
A CPU cache is a cache used by the central processing unit of a computer to reduce the average time to access memory. The cache is a smaller, faster memory which stores copies of the data from the most frequently used main memory locations.
What is the difference between L2 cache and L3 cache?
Lets start with this:
I think latest SMP processors uses 3 level caches so I want to understand Cache level hierarchy and their architecture .
To understand caches you need to know a few things:
A CPU has registers. Values in that can be directly used. Nothing is faster.
However we can not add infinite registers to a chip. These things take up space. If we make the chip bigger it gets more expensive. Part of that is because we need a larger chip (more silicon), but also because the number of chips with problems increases.
(Image an imaginary wafer with 500 cm2. I cut 10 chips from it, each chip 50cm2 in size. One of them is broken. I discard it and I am left it 9 working chips. Now take the same wafer and I cut a 100 chips from it, each ten times as small. One of them if broken. I discard the broken chip and I am left with 99 working chips. That is a fraction of the loss I would otherwise have had. To compensate for the larger chips I would need to ask higher prices. More than just the price for the extra silicon)
This is one of the reasons why we want small, affordable chips.
However the closer the cache is to the CPU, the faster it can be accessed.
This is also easy to explain; Electrical signals travel near light speed. That is fast but still a finite speed. Modern CPU work with GHz clocks. That is also fast. If I take a 4 GHz CPU then an electrical signal can travel about 7.5cm per clock tick. That is 7.5 cm in straight line. (Chips are anything but straight connections). In practice you will need significantly less than those 7.5 cm since that does not allow any time for the chips to present the requested data and for the signal to travel back.
Bottom line, we want the cache as physically as close as possible. Which means large chips.
These two need to be balanced (performance vs. cost).
Where exactly are the L1, L2 and L3 Caches located in a computer?
Assuming PC style only hardware (mainframes are quite different, including in the performance vs. cost balance);
IBM XT
The original 4.77Mhz one: No cache. CPU accesses the memory directly. A read from memory would follow this pattern:
80286 (1982)
Still no cache. Memory access was not a big problem for the lower speed versions (6Mhz), but the faster model ran up to 20Mhz and often needed to delay when accessing memory.
You then get a scenario like this:
That is an extra step spent waiting for the memory. On a modern system that can easily be 12 steps, which is why we have cache.
80386: (1985)
The CPUs get faster. Both per clock, and by running at higher clock speeds.
RAM gets faster, but not as much faster as CPUs.
As a result more wait states are needed.
Some motherboards work around this by adding cache (that would be 1st level cache) on the motherboard.
A read from memory now starts with a check if the data is already in the cache. If it is it is read from the much faster cache. If not the same procedure as described with the 80286
80486: (1989)
This is the first CPU of this generation which has some cache on the CPU.
It is a 8KB unified cache which means it is used for data and instructions.
Around this time it gets common to put 256KB of fast static memory on the motherboard as 2nd level cache. Thus 1st level cache on the CPU, 2nd level cache on the motherboard.
80586 (1993)
The 586 or Pentium-1 uses a a split level 1 cache. 8 KB each for data and instructions. The cache was split so that the data and instruction caches could be individually tuned for their specific use. You still have a small yet very fast 1st cache near the CPU, and a larger but slower 2nd cache on the motherboard. (At a larger physical distance).
In the same pentium 1 area Intel produced the Pentium Pro ('80686'). Depending on the model this chip had a 256Kb, 512KB or 1MB on board cache. It was also much more expensive, which is easy to explain with the following picture.
Notice that half the space in the chip is used by the cache. And this is for the 256KB model. More cache was technically possible and some models where produced with 512KB and 1MB caches. The market price for these was high.
Also notice that this chip contains two dies. One with the actual CPU and 1st cache, and a second die with 256KB 2nd cache.
Pentium-2
The pentium 2 is a pentium pro core. For economy reasons no 2nd cache is in the CPU. Instead what is sold a a CPU us a PCB with separate chips for CPU (and 1st cache) and 2nd cache.
As technology progresses and we start put create chips with smaller components it gets financially possible to put the 2nd cache back in the actual CPU die. However there is still a split. Very fast 1st cache snuggled up to the CPU. With one 1st cache per CPU core and a larger but less fast 2nd cache next to the core.
Pentium-3
Pentium-4
This does not change for the pentium-3 or the pentium-4.
Around this time we have reach a practical limit on how fast we can clock CPUs. An 8086 or a 80286 did not need cooling. A pentium-4 running at 3.0GHz produces so much heat and uses that much power that it becomes more practical to put two separate CPU's on the motherboard rather than one fast one.
(Two 2.0 GHz CPU's would use less power than an single identical 3.0 GHz CPU, yet could do more work).
This could be solved in three ways:
1) Is an ongoing process. It is not new and it will not stop.
2) Was done early on (e.g. with dual Pentium-1 motherboards and the NX chipset). Until now that was the only option for building a faster PC.
3) Requires CPUs where multiple 'cpu core' are build into a single chip. (We then called that CPU a dual core CPU to increase the confusion. Thank you marketing :) )
These days we just refer to the CPU as a 'core' to avoid confusion.
You now get chips like the pentium-D (duo), which is basically two pentium-4 cores on the same chip.
Remember the picture of the old pentium-Pro? With the huge cache size?
See the two large areas in this picture?
It turns out that we can share that 2nd cache between both CPU cores. Speed would drop slightly, but a 512KiB shared 2nd cache is often faster than adding two independent 2nd level caches of half the size.
This is important for your question.
It means that if you read something from one CPU core and later try to read it from another core which shares the same cache that you will get a cache hit. Memory will not need to be accessed.
Since programs do migrate between CPU's, depending on the load, the number of core and the scheduler you can gain additional performance by pinning programs which use the same data to the same CPU (cache hits on L1 and lower) or on the same CPUs which share L2 cache (and thus get misses on L1, but hits on L2 cache reads).
Thus on later models you will see shared level 2 caches.
If you are programming for modern CPUs then you have two options:
I realise that I have not yet mentioned L3 cache, but they are not different. A L3 cache works in the same way. Bigger than L2, slower than L2. And it is often shared between cores. If it is present is is a lot bigger than the L2 cache (else having it would not make sense) and it is often shared with all cores.
Lower CPU cache will result in a higher probability of cache misses, which will significantly degrade performance. That being said, another entire core will allow the computer to run at least two threads simultaneously, so it's a trade-off when you have to pick one over the other.
From the above graph, we can see that when the cache size is over 1 MB, the probability of a cache miss is already extremely low, and shows diminishing returns with increasing cache size.
Extra CPU cores, on the other hand, can show drastic speed increases when applications take advantage of the multiple cores.
For most real-world applications, the extra execution core will provide a better gain in performance over the extra cache. Both cache size and core count are vitally important when weighing a computer's performance, but when you are dealing with a relatively low amount of cores to begin with, extra cores will usually provide significant performance gains.
...
Best Answer
The L3 cache is simply another layer of cache.
Typically there are now 3 layers of cache on modern CPU cores:
L1 cache is very small and very tightly bound to the actual processing units of the CPU, it can typically fulfil data requests within 3 CPU clock ticks. L1 cache tends to be around 4-32KB depending on CPU architecture and is split between instruction and data caches.
L2 cache is generally larger but a bit slower and is generally tied to a CPU core. Recent processors tend to have 512KB of cache per core and this cache has no distinction between instruction and data caches, it is a unified cache. I believe the response time for in-cache data is typically under 20 CPU "ticks"
L3 cache tends to be shared by all the cores present on the CPU and is much larger and slower again, but it is still a lot faster than going to main memory. L3 cache tends to be of the order of 4-8MB these days.