38C3 - The Design Decisions behind the first Open-Everything FABulous FPGA

38C3 - The Design Decisions behind the first Open-Everything FABulous FPGA - https://www.youtube.com/watch?v=3Lll9_-gYGg

Foreign. So our next speaker will be Dirk and he will be talking about the design decisions behind the first open everything Fabulous fpga. Because we all like to have FPGA to do stuff really fast and please give him a warm welcome and enjoy the talk. Yeah thanks and for the kind introduction and welcome to my talk. So I'm a university guy and our group is called Novel Computing Technologies and we are mostly working on FPGAs looking into cat tools, doing a lot of work on partial reconfiguration, looking into reliability aspects like you have on a modern data center fpga a gigabit configuration data and that may be impacted by single event upsets. We do applications related to FPGAs. Today we'll be on our work on embedded FPGAs. We also look into hardware security and we have one fancy project that is putting 16,000 RISC V threads on a single FPGA. We get a performance out of more than one RISC V MIPS per lookup table. If you want to know how it's done, there is somewhere Riyadh who will join us later in a breakout. Okay, one slide on security before I really go to the main topic because there are many people working on security. Usually people try to minimize power on a chip. We ask the question the other way around. How much power could you possibly burn on a chip? Yeah, and we call this power hammering potential. And the power hammering potential on a data center FPGA that you get at Amazon Web service is something like 15 kilowatts. How do I get to this number? You can take a 6 GHz ring oscillator and you fan out on all wires and then you record your power and then you can say OK, with eight lookup tables with we can manage to achieve 100 milliwatts. And the data center fpga has 1.2 million lookup tables and that gives you the 15 kilowatts that we have. And by the way, the fastest switching activity we managed was 8 GHz. Then we are trying to fry these chips and we are ramping up the power. And interestingly that high power density which is in the same order of magnitude and the heat density on the surface of the sun. The chip survives 25 watts. But we are not giving up. We saw within one week of 5% aging. But we are also working on hardware trojans directly at bitstream. But anyway, back to the main topic. Short introduction into FPGAs for the people not so familiar with this and how you compare it maybe to a cpu. This is the idea of a fundamental von Neumann model. So you have your CPU somewhere and your alu, and you get an instruction stream in that tells you what operands to fetch, what, what operation to do, maybe where to write your result back. And the important thing is that you have a relative expensive memory operation to get your instruction in with each cycle and you have some effort to decode, and that is costly. So in order to amortize the cost, people started using simd. So instead doing one load, you do four or eight or whatever, many loads and corresponding multiplication and so forth. Yeah. And then maybe we have billions of transistors on a chip, we do copy and paste kind of. Yeah. And that is kind of super simplified. The model of a GPU on an fpga we could play the same game, but usually what we try to do is like more a data path. And the way it works is that what we do here in the CPU side, in a sequential execution, we basically unroll in space on the chip. Then instead of a load instruction, I put maybe a DMA unit in, and instead of a multiply instruction, I put a multiplier in and so forth. Or if I have an instruction like a rotate, that's the thing in the middle. This may be just wiring. So I would not even count this as an instruction in the FPGA case. And then the idea is that you operate this more like an assembly line. So if say the multiplier puts its result out and it goes into the adder, you will always start with the next multiplication. A way to look at it is also that we can say on a cpu, if you program, you program against a given architecture or instruction set, right? On an fpga, there is no such thing as an architecture. You have to do the architecture in the first place. And that allows you to do an awful amount of optimizations. However, all this programmability comes at a cost. And this is why you may want to customize what you want to program. And a nice way to visualize how FPGAs actually work is you only put in your chip what you need to solve your problem. You get the idea. Okay, how do FPGAs work? Well, the basic function generator is a lookup table, the most trivial thing that you could imagine. So you have a truth table and the output you may take out to build bigger combinatorial path, or you may go through a flip flop. Yeah. And if you want to implement, let's say a NAND gate or an and gate. Here, the green one is an and gate. You just fill out your logic table and you say I have zeros everywhere except for the last entry where all my inputs are one, I get to one out. The red one is an example for an or. Okay. The way this is implemented, this is shown in the left side of the figure, is basically just a large multiplexer. And the blue boxes that you see, those are the configuration bits that you will program if you upload your bit stream. And then in front of your lookup table, you have again, multiplexers. But the difference is that inside the lookup table, you see that the blue configuration boxes, these latches, they are connected to the data input to the multiplexer. And for the routing, it's connected to the select inputs to the multiplexer. But the message I want to do is that regardless if it's logic or routing, fundamentally FPGAs are made of huge multiplexes and an awful amount of configuration latches. That means if you want to optimize something, you will actually look into optimizing multiplexes. And those ledges. And that is relatively simple, but gives you most of the efficiency gain that you can get. And then you compose a big fabric out of that. This is actually a screenshot from within the tools. So with our fabric toolchain later you can do bitstream generation and then you can load your netlist in this editor and you see actually how it's physically implemented on the chip. Yeah, it looks pretty familiar to the tools from Synyx, Itera, whatever FPGA vendors you use. And we have different blocks. The small squares or the big square that you see is a switch matrix. This is where all the routing is condensed. The small blocks is like logic blocks. And then we have different columns. Some are for I O, some are for logic. The red columns, what you see in the very right of the small fabric, this is memory. The green ones are DSP blocks for arithmetic. But you get the idea. Okay. Whenever you start a new project, I think you have to justify why you do this. Sometimes it's for the sake of curiosity to do this, but here I had really a reason to do this. And the origins of my project, of this project comes from a major project where we wanted to make reconfigurable chips for memristors. So we got in the UK 6.3 million to investigate memristor technologies to build reconfigurable chips. And our task was, let's build FPGAs out of memristors. Yeah, and memristors are basically analog tunable resistors that you can have. And these resistors, they need high voltage to be programmed. Then you have a resistive state. But at the end of the day we want to have zeros and ones. So we need discriminators, we need some programming logic, voltage shifters and so forth. So we did all the heavy lifting engineering for our first chips. Then we also start doing the first version of a chip. This was done with industry tools, but this was showing us that we can actually build FPGAs. And in order to do such work, we need an ecosystem. So we need something that generates the FPGA fabric, but also we need something that allows us to program that thing at the end of the day. And there is a tool chain out VPR done by the University of Toronto. That thing is more than 20 years old and that's kind of the gold standard. And it is a good tool. Nothing to complain about it, but it's difficult to control it. And I'm a little bit of a control freak. I want to have full control over the fabric aspects. So this is why we did something on our own. There are other open source FPGA projects like Open fpga, there's a project from Princeton, they are all based around dpr, but they are not very optimized, like in terms of area for, for example, and I didn't like that one because at the end of the day we want to show that we can eventually gain a benefit. Right? So we need reasonably good quality of results to get credible results. Yes. So it was basically time to do something new. And we called this fabless. So what is Fabless doing? Well, it's a fully integrated framework for building embedded FPGAs. And here in the top, this is fundamentally the code generation. So at the very top you specify your fabric. It's basically just a few CSV files that you throw in. We have a primitive library, for example, with lookup tables, arithmetic blocks, these kinds of things. And then Fabless generates you the RTL code, some constraints, but also in the middle, an architecture graph that we need later for place and route. Then in the bottom we are running a backend flow to generate the actual chip. And the output is so called gds. That's the file that you send to your fab and that contains basically the information for the masks that you have. And on the right hand side in the figure, that's actually the bitstream compilation flow. And most of that work is actually done by other projects. And also for the compilation we can use industry tools like from Cadence, but there's also open source tools like openlane. And openlane is in these days incredibly stable and allows basically civilians to make their own chips. Okay. We aimed to make our framework pretty versatile. That allows you, for example, it allows you to define awkward shapes like in this case, it's like an inverse T shape. You can put in your own logic or DSP or whatever. Tiles, you can have custom tiles. For example, if you want to do something specific for machine learning or let's say post quad Encryptor, we have some users that exactly play that game. You can do this. Or if you want to adjust the routing, you have also possibilities to do this. So think of that. You have a sensor here. You do some processing until it goes into a memory. So you may have a data flow in one direction, so you may need less wires in that direction. And that can help you eventually saving things. If you want to do FPGAs or start doing FPGAs, somehow you are a bit crazy or mental in the sense that this is a figure that EE times composed once. This is about all the FPGA starts up over a couple of years in this and the list is by far not complete. And the vast amount of these companies failed at the end of the day. So there must be something that's not so easy and a common pitfall. Sometimes these companies do architecture that are inherently flawed. For example, there was a company called Tabula. What they wanted to do in the architecture could never ever work. But there are other companies that usually miss out on the tool aspects behind. So if you want to make FPGAs the tools to do for example, the bitstream compilation is the much harder problem than doing actually the silicon on its own. One comment a little bit on let's say reflection on this one is that unfortunately a lot of these FPGA companies do chips that are used for military purposes. And I see a lot of people that are happy about Govin, which is a cheap Chinese fpga. It's basically a lattice clone and they have pretty good ties or tight ties with the Chinese military. So keep this in mind if you work on these chips. But anyway, back to the design decisions. So if you want to make we know that now lookup tables are the fundamental function generator and we have to decide how big should the lookup table be. And this was a large body of work that had been done in this direction and people worked out well, let's take a 4 input, a 5 input, a 6 input lookup table and see how well this actually works and how well they are utilized. And the idea is that I say, okay, if I get more bigger lookup tables, I may need less of them, but quite often I will not utilize all the inputs right? And that leaves me something's unused. And what this table actually shows is that for the different colors here, it's like the blue ones is for lut4, the red ones is for lat5, and the yellow for lat6. And what it shows is that if you go for large lookup tables, you will not even use half of them, or something like one third only will be used at its full capacity. So it looks like that having large lookup tables is maybe not the very best idea. However, large lookup tables have one advantage. They are usually better in power and latency because you have less levels of logic on average, and that helps you to save something. However, when you buy an fpga, you think you buy lookup tables or logic, but in the end of the day or at the end of the day, you pay for the routing. So it's nice that you have all these logic blocks that you can wire up together, but this is really expensive. On an asic, I just draw basically a metal wire from A to Z. On an fpga, I have to go through all kinds of switches to get there. And 70 to 80% of your chip area is dedicated to the routing. It's that expensive. So if you go to Xilinx, Altera, Lattice, you name it, you think you buy lookup tables. Well, you pay for the routing. Yeah. And so. So you have to be very mindful in deciding what your routing fabric should look like. And one thing that you do is a simple observation that in a netlist, if you build hardware, you have some components that are tightly connected and some components that are more loosely connected. And you try to reflect this in your fabric by clustering some of these lookup tables together, giving them, let's say, say, more denser connectivity, and then have maybe a little bit less connectivity from cluster to cluster. And this also was research that had been done before where people tried out different lookup table sites and different cluster sizes. And the sweet spot is the one that is circled in the middle, that is that you take four lookup tables and you group something like four to 10 together. And this is exactly what we did in fabulous. However, if you would go to a more advanced process node, let's say 22nm these curves, they move a little bit and they move a little bit towards large lookup table sizes. And this is why, for example, Xilinx Altera these days use lat6. That's not the only reason, but that's one of the major ones. This is an interesting one. So fundamentally it looks like, well, it's just multiplexes, right? But if you want to implement this on a chip, there are sometimes constraints you have to obey and you should obey. It's not mandatory. But what this figure shows you here is that if you want to implement a wide multiplexer, you can't just take 16 input multiplexer. It's done over multiple levels. And in this case it's shown for two input multiplexers. And you see, I do this over three layers or levels. And what I want to show here is consider the last green level. If, for example, the top path would be faster than the bottom path, then your latency would depend on this input two that you have on the green input. Okay? And that is a nasty problem if you do chip analysis because you would. Your timing has to be aware of the logic state. And this is not how we usually do timing. You could compute this, but this is a nasty one. You don't have to understand all the details. I just want to make the point that just throwing random gates on the chip to get a high quality fabric is sometimes not enough. And we did actually a tool together with somebody at Berkeley that implements these things automatically in a tool. And the way it works is that you have kind of a recursive structure to implement all these figures. Okay, then I may want to implement bigger lookup tables. And one way to do this is that I can take adjacent lookup tables with a multiplexable bigger ones. Another way what people do is that they take what we call fractionable lookup tables, that you build large lookup tables and you put them into pieces. Again, this is what ALTERA is actually doing. However, if you look at lookup tables, the input routing multiplexes, they're actually more expensive than just the lookup tables here. And I put this figure in because it looks cool. But I tell you what it is. Yeah. What you see here is this is for older Xilinx FPGAs, they have eight lookup tables A to H. Each lookup table has four inputs. And what I have shown is like how many inputs that go into my switch matrix. So I have a switch matrix, I have a wire that comes in, okay. And that goes to eventually one lookup Table input and it goes maybe to another lookup table input. And if they share this port, I count this, okay? And so on the diagonals, that tells you how many inputs I have on that specific multiplexer. And what it shows is that basically the lookup tables for A, B, C, D, they share exactly the same inputs. Okay, so that observation is something that we used to reduce some cost in the routing. Okay, but this is low level details. Another important thing is that if you build, for example, arithmetic, you usually use something like carry chains. It's like a carry ripple adder that you may remember from your foundations in computer engineering, where you have like a full adder and it goes from one cell to the next cell to the next next cell. You ripple through a carry. And this is what actually lettuce is doing in front of their lookup table. They have like this carry logic in front of the lookup table. And while this works, it only allows you to do addition fundamentally. And I will skip this one for time reasons that I don't run short. That's not so important. Xilinx, however, they do it in a different way. They do the carry chain after the lookup table. It's not important to understand all the details, but if you do this after the lookup table, you can also do fast arithmetic. But you can also use this to implement wide logic gates. Sometimes you have, for example, wide comparators, you have wide and gates or gates, these kinds of things, and you can implement that one. An interesting one is that Xilinx, they fundamentally keep the principle of operation in their logic. What they did since, let's say 20 years ago is fundamentally the same thing that they do today. The only thing is that the lookup tables have become bigger. So these eight blocks that you see highlighted there that are those lookup tables, then you can combine multiple lookup tables to form bigger multiplexes through those multiplexes. Bigger lookup tables, and that gives you larger lookup tables. Okay, to summarize this, there is not really a winner. So all these different FPGAs, and they haven't mentioned everything here they have their individual pros and cons. And the real point I want to make with everything here is they fundamentally all work. And the three most important things, if you want to do FPGAs, is number one, tools, number two is tools, and number three is, well, tools. So when we did our design decisions, we actually looked, what are this, what support do we get from synthesis tools? What can we do with place and route? And we made Decisions accordingly. Then I mentioned routing is the elephant in the room. And what this figure shows you is the adjacency of Xilinx FPGAs. And what does it mean? Each dot that you see in this picture is a possible programmable connection on the columns. This represents one multiplexer. And all the horizontal that gets in that are the inputs into my switch matrix. Okay. What this means is like I have this large block that does my routing and then I have wires that come from, let's say, other clusters. Right. And that would be my inputs. Okay. And then I have maybe lookup table inputs or wires that go to other ones. That would be my columns and so forth. Yeah. And this is shown here. And what the figure should express here or what it shows you is that the routing is incredibly sparse that you have. And so point one, point two is also what this shows is how much of the routing is lookup table inputs, how much of the routing is for adjacent wires. I don't drive you too much into the details here. I'm very low level. I know, but the message is that, that an awful amount of routing is just the inputs to your lookup tables. That was this figure with these blue boxes that you saw. And so if you implement such a routing fabric, you will have to be a little bit mindful with designing these switches. And we almost get similar density for our routing here. A few chips that we did. There should be a video. The video should be. Actually that was the first bigger fpga. Maybe you can see it a little bit. But this was like the first major FPGA that we did. And it's this street animation. It uses DSP blocks. The bitmap that you see is stored on internal block ramps. We even do a square root for the rendering in the logic and you get basically all. All the elements that you get in a modern fpga and they work together. We had an interesting bug with that chip. And the way you do the clock distribution is shown there that you have like a central clock point where you pump your clock in. And then it's distributed from tile to tile to tile to tile. Okay. And what we did differently is that the block ramps, these are the large blocks at the right that you see in this chip. They got their clock through a different clock tree. And there is like a little bit of a phase relationship between the clock that we had in the fabric versus these block ramps, which ended up in having hold time violations, which is actually a nasty thing, which you can compensate with the routing tools. And it took us something like. For the demo that you saw, it took us something like 100 times routing. Before we got a demo that was working. It couldn't be done in a more structured way that we can do today. But when we did this demo, we took something like 100 times to get it routed. This was our first almost everything fpga. I say almost because there was one tiny step that we couldn't do with open source tools and we actually had to use an industry tool, Innovus. And however, this was done in a skyward 100 nanometer process. So it comes with an open PDK and you can basically download this, go on the git repo and you can explore the chip all the way down to the transistor geometry. Yeah. Yes, we did another chip, this is done entirely with open source tools. If you check out that git repo, it compiles through all the way to the final fabric without any user intervention. And it's a tiny FPGA for today's standards. Yes. You have an idea Data set FPGA is more than a million lookup tables. We are not getting even a thousand. So it's to some extent tiny, but you have to be aware that it's just something like 10 square millimeters area that we have here. And we are talking about 130 nanometer process. Okay. And if you want to investigate this, that's the git ripple. The next step is we don't have a proper timing model for that one. So if you do place and route, we look a little bit in the crystal pipe if it works. But we know that we can do better. So where there's some homework to be done, we have to do a little bit more on the physical constraints to automate this. A couple of more optimizations. We want to put in more tape outs. So we want to do Global Foundries 22, maybe TMC, EHRP and so forth. And we want to do actually an open everything RISC V plus embedded FPGA board. This is like, oops, it's in a audio cassette packet. So this is like the first open everything chip that we did. Yeah. And we want to develop this a little bit further and put RISC V core in with an embedded fpga. And one thing, I'm not so sure what I should do. Where I will ask you what you would prefer is we can take memory macros that have very poor performance, very poor density. So they give me only like 12 megahertz and they give me only like 1 kilobyte or I can use Siemens macros that are not open where I get two to three times the capacity and 100 megahertz. What would you prefer? Open everything and sacrifice on performance and density. Or would you like to have the industry macro so you can download the chip then later but the memory macro would be a black box for you. So what would the audience prefer? Who would go for open everything? Okay, actually I climb to that. Anybody on the industry side? Okay, I guess that is a clear vote home. I have to stop here. If you want to look a little bit more you can scan the QR code on my logo here. That gets you to the documentation website. More on that one. We actually run a summer school FPGA Ignite in Heidelberg. The chip that you see on the right hand side is a. Sorry. During this summer school we do actually chips that get taped out and participants get them back. This was like a chip that we did. If you are interested in doing such things and make your own chip, stay tuned and go on the website. Thanks to all the people that make this research actually possible. And one final thing I skipped this one is that we have a meetup later on open source FPGAs and open ASIC design meetup. It's at 4 o'clock today, close to the cows post. So see us there if you want to see more. Thanks for your attention. Thanks for the talk. Do we have questions? I see a question. Let's start there. Is this on? You asked which we wanted the the denser SRAM or the open sram? I want both. The Siemens cells are denser because they break the foundry rules. They use the special rules for SRAM cells that violate the other design rules. I think you should have someone design your own push design rules, high density sram. But that's just me. I wanted to thank you for bringing back partial reconfiguration. That's something the industry has ignored for many many years because the defense customers don't care about it. So I'm very glad to see that we now have FPGAs that we can do partial configuration research on. It really is the killer app for FPGAs. My question for you is not so much about the technical material, but about the naming in the software world. The definition of free software includes three fundamental freedoms and one of those is to modify any part of the software. In order for software to be considered free software open source, you must be able to modify any part. On slides 31 and 33 you had two die photos of chips. And I noticed quite distinctly in those die photos the inclusion of the Caravel management engine at the bottom edge of this die photo and at the far right edge of the slide photo on slide 33. I know this because I for a very long time was targeting Skywater 130. Unfortunately, Google requires the inclusion of this management engine on every chip that goes through their tape outs. And the management engine sits between your design and the outside world with the ability to man in the middle every single pin. So my question for you is if the inclusion of this Caravel management engine is mandatory and it cannot be modified, does this really qualify as AN Open everything fPGA? Yes. So what you see there is not considered. That's just the pat ring. So there is no Caravel at all. Is this IHP or Skywater? That's Skywater. So they're allowing tape outs without the management engine. Now we did a paid one at the Google. The Google program is dead anyway. Yeah. So that tape out costs 10 grand that you have an. I asked them for this three years ago and offered them more than ten grand and they wouldn't do it. So I'm glad to see they've changed their minds. And good news is you will get way more iOS. So there's something coming up that gives you over 100 iOS. Actually the configuration of the I O pins is done through the FPGA fabric. So wire configuration bids out and you can configure the I O pins so it almost feels like what you get from the big vendors. Thank you. Next question. Same microphone. These are microphones nobody likes. Yes. I have a question about the timing analysis. Do you also want to design your FPGA or actually the. The routing part so that you can also model the stray capacitance and inductance of your nets? Yes. So what you have to do is it's basically a spice simulation to get the exact arrival times for your clocks and then you want to have. Basically you have to query your net list for all the let's say multiplexer to multiplexer or multiplex flexitool lookup table wire segments. Yeah, in reality it's a bit more complicated because if you do pass transistor multiplexes then switching on multiple paths may change like slew rate fan. So different fan outs have impact on that one. So that has to feed all back into the model. Yeah. So but it's. But that is exactly the part that we are missing. So that you can do full blown place and route. Yes, please go yeah, so thank you professor for the. For explaining all this. This is all proprietary information most of the time. And how these FPGAs work, we don't know. We just use it as a user. So this was very enlightening. My question would be about the slide where we. You showed the cpu, the GPU SIM and. And then the data flow of the fpga. I think it was, yeah. So my question would be now why would these vendors or you also later would also want a soft cpu, RISC V or nfpga? What would be the use case when you have literally built your own pipeline here? Right. Okay. I call it embedded FPGAs because reconfigurability is expensive and you only want to put it in where it helps you. And if you need a cpu, it will be a hardened cpu and you would put the fabric beside that to just do the things that have to be reconfigurable. So if you want to. Like the last thing that I was referring to, this box that we want to do is. Will be hardened core. Yeah. And we use. Actually the fabric is different, designed to also do embedded CPUs or soft CPUs. And the reason for that one is I just told you the most useful thing to run on an fpga. Embedded FPGA is a cpu. And everybody asks me, have you done this and this and this CPU on your fabric? And they will only say your fabric is useful if you can run this and this and this cpu. Well, knowing that this is something you would never really run, run on that one. Ergo, we put some support in like distributed memory for a register file or so to accommodate that one. But this also expresses that you can tailor the fabric to your need. So if you do like let's say just hardcore arithmetic, you can put in more DSP blocks or specialized DSP blocks or Tensor blocks or you name it. Thank you, Michael, for number one. Yeah, thanks for the talk. It was really enlightening. My question is regarding the tape out, you said that you will bring up a little box. Is there some way to get notified when this is coming up? And also to get not only the modules, but maybe also the bare chips to implement in own designs. Not sure if I get your question right. Is it related to the summer school or. Because we are not a chip broker or anything like we run these things basically in house. To be fair, we used efabless and they have a relatively streamlined service. You submit their gds and they do the fabrication, the packaging and you even get like these white thing, if you see it from the distance, they even sold the chips on a small breakout thing and you get even a pcb, that's our PCB that we made. But you get something from them back that's kind of streamlined I think entry price. I don't want to make advertisement for them. I hope that there will be more vendors coming up with this one. But that's like a 10 grand entry level price that you pay. There's cheaper options like tiny tape out and if you want to get it for free, join us in our next summer school. Okay. Yeah. The question was more than regarding if there's any tape out bayou planned for. If you have something, we may find some space for you. If you have a good idea, we may just throw it in. So. Yes, ping us. Thank you. Microphone 3 hi. First of all, thank you for the talk. You mentioned memory stores and I would like to ask you do you see potentials for spintronics technologies like MTGAs to improve FPGAs design like in density, efficiency or reliability? Yes. Thank you. Okay, so hold on. Yes. So one thing where it may fly actually high is if you recall the multiplexer tree for my lookup table. If I can store basically in an analog form 2 bit in each cell by using an RM cell, so I use it in analog mode then I need only half as many cells. My multiplexer tree is only half the size, so that's great. However, now I have the problem to discriminate four states into two bits. That costs me something. Okay. If you can do this cheap, you will win. And we found a way to do this cheap. It's still research. Okay, but this is one of the games for example that we want to play. Reliable is Aram Technology is in theory inherent reality reliable in the sense that it's not. It's resistant to single event upsets. However, the technology itself is headroom for improvement. Yes, if this answers the question. Thank you. Unfortunately we are at the end of our time slot but I'm sure you can still talk to our speaker afterwards so please give him another round of applause.