Tensor Processing Units (TPU) for Consumers; The next big thing?

[beginner99]

This will just be a fad and in couple years go the way of the dodo like physx.

It's also kind of a contradiction, the better the card the higher the resolution you can play at and the less you need AA. I doubt we need AA at all at 8k.

 

[mv2devnull]

It does ask the question of what happens when you get AMD AI cores, or maybe google or ARM AI cores in phones. If you have to train the network then how cross compatible is that training?

Huawei Honor 10 phone's brochure boasts:

The AI Camera Recognizes Multiple Subjects and Applies Localized Optimization in One Image

Cutting-edge AI algorithm instantaneously pairs subjects with their optimal camera settings according to the context of the image.
Faster and more efficient AI algorithm performance backed by an independently built-in Neural Processing Unit (NPU).

Sounds like Huawei has AI cores and a trained model already. At least they are no worse in sales speech than NVidia.

The "network" is just a math expression. Not really different from y=a*x+b.
You decide the value of x, evaluate expression, and get the corresponding value for y.
The training was with known (x,y) pairs. The training did compute the equation and the values of constants (a and b).
All you need to copy from trained model to your chip is the formula and the values of the constants.
(Okay, the y=a*x+b is a very simple model.)


As for the TPU's being "locked" ... how would you implement a lock on consumer cards?

 

[NTMBK]

This will just be a fad and in couple years go the way of the dodo like physx.

It's also kind of a contradiction, the better the card the higher the resolution you can play at and the less you need AA. I doubt we need AA at all at 8k.

Just because you can rasterize at high resolutions, doesn't mean you can calculate all your effects at high resolutions. Maybe you perform ray-tracing effects at 1080p, then use a DL upscaler to combine it with 8k rasterized image.

 

[Guru]

Seems to me like Nvidia has a lot of failed graphic cards from their pro models, so they are essentially cramming all of these things into consumer products from the failed big dies.

I don't see much benefit to these and the RT cores, I'd rather get cheaper graphic cards with more raw performance.

Is Ray Tracing the future? Yes, its is the FUTURE, NOT the present! We are at least 5-6 years away from real ray tracing in real time for the consumer space. Realistically probably even more than 6 years, I'd say 10 years for high level ray tracing in real time for hundreds of millions of rays.

Are RT cores okay for developers NOW? Yes, they are useless for consumers though! You get mildly better light at a huge performance penalty, essentially like what gameworks is for Nvidia, minor visual benefit for a huge performance penalty!

 

[TheF34RChannel]

There's not much love for the new cores I see. I beg to differ; who knows what workloads they'll concoct for the tensor cores to benefit this or that. I think new tech advancements were much needed.

 

[PeterScott]

I'd rather get 35% performance uplift, new RT cores, and new TPU cores, than just a 40% uplift and no extras of typical release.

If this were just a typical performance uplift release, I barely would have batted an eye. As it is there is a wealth of new potential.

 

[TheF34RChannel]

I'd rather get 35% performance uplift, new RT cores, and new TPU cores, than just a 40% uplift and no extras of typical release.

If this were just a typical performance uplift release, I barely would have batted an eye. As it is there is a wealth of new potential.

Precisely!! We've been banging on the same performance increases, and those alone, on more or less the same architectural design forever.

I'll bet similar things were believed when Cuda cores were first introduced. New introductions give way to apprehension I suppose. And without any substantial test results and being on the brink of a new era we are kept in a holding pattern. It doesn't keep me from seeing its potential - or fantasizing about it.

 

[beginner99]

I'd rather get 35% performance uplift, new RT cores, and new TPU cores, than just a 40% uplift and no extras of typical release.

If this were just a typical performance uplift release, I barely would have batted an eye. As it is there is a wealth of new potential.

I would rather get the 40% uplift on a smaller die for a reasonable price or said otherwise:

A meaningful performance/$ increase

The fundamental aspect of new manufacturing process and uArch are to get more performance for same amount of $. Keeping performance/$ constant means 0, absolutely 0 progress. And it actually hurts NV because there is no point for most users to upgrade.

 

[Guru]

I'd rather get 35% performance uplift, new RT cores, and new TPU cores, than just a 40% uplift and no extras of typical release.

If this were just a typical performance uplift release, I barely would have batted an eye. As it is there is a wealth of new potential.

That is a false equivalency, who said you'd get 40% more at same everything, but 35% and extra stuff?

I'd rather get 30% more performance for a much lower price. 2070=1080 performance for $400, 2080=1080ti for $550, 2080ti 30>1080ti for $700

But if they didn't have all these advertisement cores and instead it was standard cores packed in a 700mm chip, they could have Volta level of cores AND MORE and if they spent time optimizing the architecture and optimizing drivers, it might have had 50% more performance or more!

This way we get over expensive cards that barely offer 30% performance increase and that is according to Nvidia's own numbers, we might actually get lower scores when reviewers get a hold of these cards.

 

[arandomguy]

There's not much love for the new cores I see. I beg to differ; who knows what workloads they'll concoct for the tensor cores to benefit this or that. I think new tech advancements were much needed.

I feel you are misunderstanding the prevailing sentiment. The issue people are having with the "new cores" is that it is being linked to the substantial price increases in each segment. The market (or at least online tech enthusiast demographic) has been conditioned to substantial perf/$ increases with GPUs as time goes on with each new generation as well as the new technology (features) as an additional bonus.

Consumer's aren't going to look at the "new cores" in vacuum if it is something they are going to need to pay for. I'm sure in some eyes given expected perf/$ increases they may be viewing those "new cores" of adding even something like 50% to the price they will need to pay. So to them it's a question of whether the new technology is worth 50% more.

 

[PeterScott]

For those intent on complaining overmuch about the "lack" of performance uplift, perhaps a little recent history.

This article about Generational GTX performance has a graph of generational improvement. Of the last 5 generational cycles, only 1 had more than 30% uplift, which was GTX 1000 series, which had the biggest uplift since GTX 280, way back in 2008. So GTX 1000 may have spoiled people a bit with expectations.
https://www.dsogaming.com/news/nvid...has-dropped-from-60-pre-2010-to-30-post-2010/

GTX 480->580 14.9%
GTX 580->680 28.8%
GTX 680->780 20.4%
GTX 780->980 29.9%
GTX 980->1080 56.3%

GTX 1080->2080 ??? Looks like 30% to 50% depending on game, and 80% or more when using DLSS vs older high quality AA.

So the 2000 series will have Better than Average performance uplift, even in OLD games, to go along with a wealth of new features.

The issue people are having with the "new cores" is that it is being linked to the substantial price increases in each segment. T

Most of the price/performance complaining willfully ignores that the 2080Ti is really this generations expensive early adopter Titan card with a name change. In the previous two releases you did not get the new big die card for cheap early, instead it showed up as a Titan costing $1000-$1200. Factor in the name shift and it's exactly the same price as in previous releases, and modest increase at the lower tiers...

It's too bad NVidia didn't call it Titan, as it would have kept the focus on the performance and technology, instead a lot of people just lost it over what amounts to a name change.

Pretend for a moment, that 2080Ti was called Titan T, and tell me what there is to get outraged over?

 

[Qwertilot]

Titan probably wasn't available as that's seemingly now part of their 'compute' GPU line for prosumers - so Titan V, Titan son of Volta etc. That's actually what Titan was originally (allegedly?) for before Titan P cropped up so actually commendable for a change.

2080ti wasn't a smart choice though.

 

[arandomguy]

Most of the price/performance complaining willfully ignores that the 2080Ti is really this generations expensive early adopter Titan card with a name change. In the previous two releases you did not get the new big die card for cheap early, instead it showed up as a Titan costing $1000-$1200. Factor in the name shift and it's exactly the same price as in previous releases, and modest increase at the lower tiers...

It's too bad NVidia didn't call it Titan, as it would have kept the focus on the performance and technology, instead a lot of people just lost it over what amounts to a name change.

Pretend for a moment, that 2080Ti was called Titan T, and tell me what there is to get outraged over?

In general I think debating this in terms of the actual product releases at this point is a bit pointless as the actual facts (confirmed numbers) are not in public hands. As well as the semantics argument over what each segment is further clouds the issue.

If we want to isolate it the discussion into the value of the "new cores" it is perhaps better to frame it into a hypothetical at this point. Given your price segment how much more of a premium do you (or whomever else) actually place on the new cores?

It would be interesting to see the contrast on this value between the "pro" group and the "nay" group.

Eg. Given a $500 GPU how much more would you pay for the "new cores" if everything else were identical?

 

[Geegeeoh]

GTX 1080->2080 ??? Looks like 30% to 50% depending on game, and 80% or more when using DLSS vs older high quality AA.

"Depending on game" you have GTX 780->980 even up to 60% faster and GTX 980->1080 in the 60-70%+ range.
https://www.hardocp.com/article/2018/07/25/nvidia_gpu_generational_performance_part_1/17

But you forgot that the main point is the little dollar sign: it's not just more perf it's better perf/$.

 

[PeterScott]

But you forgot that the main point is the little dollar sign: it's not just more perf it's better perf/$.

You forgot to read the second half of my post.

 

[Geegeeoh]

Nop, I read that… I don't get how "big die card for cheap" (funny, if not delusional) of the TI~Titan (cheap on letters? is that it?) changes anything.

Old 600$ card is now 700$, but good news everyone: the 1200$ is 1000$... ** that somehow has changed name with the old 700$, go figure **

 

[darkswordsman17]

Just because you can rasterize at high resolutions, doesn't mean you can calculate all your effects at high resolutions. Maybe you perform ray-tracing effects at 1080p, then use a DL upscaler to combine it with 8k rasterized image.

The thing that gets me though is that isn't this upscaling "cheating"? I've seen a lot of PC gamers say consoles suck because they cheat (using stuff like checkerboarding or when they render at sub 720p/1080p and then upscale) that suddenly are touting this as the next great thing. Why all of a sudden is using such tricks totally a-ok on PC, on freaking $500-1000 graphics cards no less? Shouldn't that DL stuff be used to push quality beyond what you could render on the fly and not just good enough that you can't tell? Heck, couldn't you accomplish the far superior graphics by just having one of Nvidia's clusters render the game a ton of times at really high quality, from every angle, and then pre-render those graphics (but they'd still be alterable, as you could render how things look when changed as well?) and leave the local GPU to just render stuff like characters which it could then do at much higher quality since its not actually rendering the rest of the scene much (like it could do some simpler geometry and they apply a high res texture wrap, that could maybe include geometry for when something changes like an item gets broken it'd just pull the pre-renderd version that correlates with how it looks from that angle). Stuff like that would likely mean increased graphics along with good performance and/or being able to render more characters/interactive objects onscreen.

 

[IllogicalGlory]

For those intent on complaining overmuch about the "lack" of performance uplift, perhaps a little recent history.

This article about Generational GTX performance has a graph of generational improvement. Of the last 5 generational cycles, only 1 had more than 30% uplift, which was GTX 1000 series, which had the biggest uplift since GTX 280, way back in 2008. So GTX 1000 may have spoiled people a bit with expectations.
https://www.dsogaming.com/news/nvid...has-dropped-from-60-pre-2010-to-30-post-2010/

GTX 480->580 14.9%
GTX 580->680 28.8%
GTX 680->780 20.4%
GTX 780->980 29.9%
GTX 980->1080 56.3%

GTX 1080->2080 ??? Looks like 30% to 50% depending on game, and 80% or more when using DLSS vs older high quality AA.

So the 2000 series will have Better than Average performance uplift, even in OLD games, to go along with a wealth of new features.

Why are you comparing the performance uplifts based solely on the top level SKU name? You're the one that emphasizes how important the low level details, such as die size and power consumption are. 680 and 780 are the same gen, 480 and 580 are nearly the same die, 780 and 980 aren't particularly comparable given the different market segments they occupy. 1080->2080 is GP104 vs TU104. Restricting comparisons to match that, the 1080->2080 uplift looks more like average to below average.

(all TPU 1440p/1600p comparisons)
560 Ti->680 88.6%
(1200p) 560 Ti->680 69.4% (probably more fair)
770->980 51%
980->1080 66.7%

Although it is notable that the comparisons in the former two reviews are actually not accurate because TPU was still using the arithmetic mean then, rather than the geometric mean, but they're likely not off by more than a few percent.

Most of the price/performance complaining willfully ignores that the 2080Ti is really this generations expensive early adopter Titan card with a name change. In the previous two releases you did not get the new big die card for cheap early, instead it showed up as a Titan costing $1000-$1200. Factor in the name shift and it's exactly the same price as in previous releases, and modest increase at the lower tiers...

It's too bad NVidia didn't call it Titan, as it would have kept the focus on the performance and technology, instead a lot of people just lost it over what amounts to a name change.

Pretend for a moment, that 2080Ti was called Titan T, and tell me what there is to get outraged over?

The difference is that everyone considering the Titan always knew that there was going to be a much more fairly priced release of the same chip later on. Yes, it's likely that there will be price cuts on the 2080 Ti eventually, but is it going to be going for anywhere near $700 a year from now? Doubt it. I don't believe that there will be any more cheaper TU102 SKUs in the future. Maybe a more expensive fully-enabled Titan SKU at best. What implication does this have for pricing on future flagship SKUs?

 

[PeterScott]

Why are you comparing the performance uplifts based solely on the top level SKU name? You're the one that emphasizes how important the low level details, such as die size and power consumption are. 680 and 780 are the same gen, 480 and 580 are nearly the same die, 780 and 980 aren't particularly comparable given the different market segments they occupy. 1080->2080 is GP104 vs TU104. Restricting comparisons to match that, the 1080->2080 uplift looks more like average to below average.

(all TPU 1440p/1600p comparisons)
560 Ti->680 88.6%
(1200p) 560 Ti->680 69.4% (probably more fair)
770->980 51%
980->1080 66.7%

Yeah, some things NVidia calls a new generation are minor upgrades. That's the point. You don't just get to exclude the ones that suck to change the outcome.

The difference is that everyone considering the Titan always knew that there was going to be a much more fairly priced release of the same chip later on. Yes, it's likely that there will be price cuts on the 2080 Ti eventually, but is it going to be going for anywhere near $700 a year from now? Doubt it. I doubt there will be any more cheaper TU102 SKUs in the future. Maybe a more expensive fully-enabled Titan SKU at best. What implication does this have for pricing on future flagship SKUs?

So what you are saying is that Early adopters can expect their card to stay on top longer? 😀

 

[TheF34RChannel]

It's a shame every thread changes into a pricing discussion - it seems on everyone's mind.

I reckon if we got no new cores and just the regular small performance increase, there would be complaining that the increase would be too small. There's no satisfying some 😉

 

[PeterScott]

The thing that gets me though is that isn't this upscaling "cheating"? I've seen a lot of PC gamers say consoles suck because they cheat (using stuff like checkerboarding or when they render at sub 720p/1080p and then upscale) that suddenly are touting this as the next great thing. Why all of a sudden is using such tricks totally a-ok on PC, on freaking $500-1000 graphics cards no less? Shouldn't that DL stuff be used to push quality beyond what you could render on the fly and not just good enough that you can't tell? Heck, couldn't you accomplish the far superior graphics by just having one of Nvidia's clusters render the game a ton of times at really high quality, from every angle, and then pre-render those graphics (but they'd still be alterable, as you could render how things look when changed as well?) and leave the local GPU to just render stuff like characters which it could then do at much higher quality since its not actually rendering the rest of the scene much (like it could do some simpler geometry and they apply a high res texture wrap, that could maybe include geometry for when something changes like an item gets broken it'd just pull the pre-renderd version that correlates with how it looks from that angle). Stuff like that would likely mean increased graphics along with good performance and/or being able to render more characters/interactive objects onscreen.

He is just saying Maybe...

There are no signs that they are doing any up-scaling funny business.

What we know that they are doing, is doing ray-tracing with a small number of Rays, which leads to a a noisy image. They then use the NN engine to clean up the noise. Is noise reduction "cheating"? I don't think so, but ultimately what I am concerned with here is quality.

What else we know their are doing, which may seem more like cheating, is train DLSS for each game. The teach a network by comparing the output from the frame buffer with no AA, against nice (expensive) super sampling AA. The run through thousands (millions?) of iterations until the AI can make really nice AA for that specific game. In one sense that is cheating, because it is a heavy per game optimization to create a super light weight high quality AA mode. But again, what I am ultimately concerned with is quality. If this works and some want to consider it cheating. IMO, by all means cheat. Go Go Gadget TPUs.

As far as the original argument that AA doesn't matter at higher res, well it's really more about PPI than resolution. Aliasing will show up more on a big 4K monitor than it will on a smaller 1440p monitor. Also Aliasing artifacts can be much larger than a pixel and can stand out in motion, so AA maintains value at 4K. 8K will require 4x the GPU power to drive. With the slow down in Moore's law, good luck waiting on that much GPU performance.

 

[TheF34RChannel]

He is just saying Maybe...

There are no signs that they are doing any up-scaling funny business.

What we know that they are doing, is doing ray-tracing with a small number of Rays, which leads to a a noisy image. They then use the NN engine to clean up the noise. Is noise reduction "cheating"? I don't think so, but ultimately what I am concerned with here is quality.

What else we know their are doing, which may seem more like cheating, is train DLSS for each game. The teach a network by comparing the output from the frame buffer with no AA, against nice (expensive) super sampling AA. The run through thousands (millions?) of iterations until the AI can make really nice AA for that specific game. In one sense that is cheating, because it is a heavy per game optimization to create a super light weight high quality AA mode. But again, what I am ultimately concerned with is quality. If this works and some want to consider it cheating. IMO, by all means cheat. Go Go Gadget TPUs.

As far as the original argument that AA doesn't matter at higher res, well it's really more about PPI than resolution. Aliasing will show up more on a big 4K monitor than it will on a smaller 1440p monitor. Also Aliasing artifacts can be much larger than a pixel and can stand out in motion, so AA maintains value at 4K. 8K will require 4x the GPU power to drive. With the slow down in Moore's law, good luck waiting on that much GPU performance.

To me it's not cheating at all; I find it a way smarter method of giving the player an equal or better top quality AA image which is much less expensive on the GPU. It's darn near brilliant! And that's just the beginning...

 

[PeterScott]

To me it's not cheating at all; I find it a way smarter method of giving the player an equal or better top quality AA image which is much less expensive on the GPU. It's darn near brilliant! And that's just the beginning...

I can just see how someone could construe it that way. You compare AA methods, and the rest of them are general purpose, and here is one, tuned exactly for a specific game. If this were a level playing field contest, that would be deemed an unfair advantage.

But it is an advantage I would would hope the games I play make use of, and I agree it is kind of brilliant. Well I guess it remains to be seen how brilliant. So far it sounds like it costs nearly no overhead, and with Deep Learning applied to the specific game, it should have a great result. Hope I am not setting myself up for a let down.

But DLSS is more important that Ray Tracing to me in this release.

 

[NTMBK]

The thing that gets me though is that isn't this upscaling "cheating"? I've seen a lot of PC gamers say consoles suck because they cheat (using stuff like checkerboarding or when they render at sub 720p/1080p and then upscale) that suddenly are touting this as the next great thing. Why all of a sudden is using such tricks totally a-ok on PC, on freaking $500-1000 graphics cards no less? Shouldn't that DL stuff be used to push quality beyond what you could render on the fly and not just good enough that you can't tell? Heck, couldn't you accomplish the far superior graphics by just having one of Nvidia's clusters render the game a ton of times at really high quality, from every angle, and then pre-render those graphics (but they'd still be alterable, as you could render how things look when changed as well?) and leave the local GPU to just render stuff like characters which it could then do at much higher quality since its not actually rendering the rest of the scene much (like it could do some simpler geometry and they apply a high res texture wrap, that could maybe include geometry for when something changes like an item gets broken it'd just pull the pre-renderd version that correlates with how it looks from that angle). Stuff like that would likely mean increased graphics along with good performance and/or being able to render more characters/interactive objects onscreen.

You set "cheating", a programmer says "optimization" 🙂 The whole history of 3D graphics is a series of clever cheats. Hell, rasterization itself is a cheat.

By the way, you basically just reinvented reflection cube maps.

 

[crisium]

Well to play along with the "would you rather" game, I'd rather have had consumer GeForce cards that followed the Titan V's shader ratio. They could still have any other shader benefit that Turing has, just the math of shaders.

So a 5120 shader GTX 2080 Ti (5376 uncut Titan T), and 3584 shaders for the GTX 2080. These consumer chips would probably have to be designed without TPUs to make it economical (GV100 is 815mm2 after all). I think TPUs have potential, and look forward to evaluating DLSS, but I'd rather have a straight old-fashioned card with those specs on 12nm and save the TPUs and RTX cores for 7nm when there's more room to work with regarding die sizes.