Oxide computer 3D rack guided tour

(explorer.oxide.computer)

197 points | by darthcloud 3 days ago

19 comments

  • jjice 3 hours ago
    I want Oxide to do so well. The product is a breath of fresh air in the era of cloud providers. As an engineer, I'd kill to get to work with their hardware.

    Not to mention that working at Oxide sounds like a modern Sun Microsystems with the ideology that team has. Highly recommend their podcast "Oxide and Friends", and their original "On The Metal" show.

    I've attempted to apply to their company multiple times over the years, only to be stun locked by the application process. Not because it's a bad process, but because I feel I'm not up to par as an engineer. Maybe one day I'll go through with it.

    • EvanAnderson 1 hour ago
      Oxide certainly sounds cool. It reminds me of when I dealt with DEC gear back in the late 90s. That stuff felt more like "real computers" than any of the IBM PC-derived drek I'd worked with. Things were actually made to work together. Configurations were tested. Firmware was made for the integrated system and the system behaved like it was meant to work together instead of being the manifest behavior of all the edge cases of all the off-the-shelf disparate components plugged-together to make the resultant machine.

      I don't need to work there (nor do I feel like I'm smart or talented enough to)-- I just wish I could work with the Oxide gear in Customer engagement, too. I don't work with businesses big enough to need it, sadly. It looks so sweet.

      This is what I think of when I think of utility-scale compute-- not racks of Supermicro / Dell / HP boxes with tiny ISA buses hiding on traces on their motherboards for "baseboard management controllers" to plug into to pretend to be PC AT keyboards.

    • Fordec 41 minutes ago
      There are two companies that I want want to work for. Oxide is one of them. Many places I'm still willing to work for, but in a more neutral way. They're just mostly not hiring for my role/location as far as I can tell, so it is what it is.
      • hagbard_c 30 minutes ago
        And the other company you want to work for is ___________ because they ____________. Inquiring minds want to know.
    • convolvatron 2 hours ago
      I've gone through the same process, not so much that I don't think I would be worth considering, but serious code and documentation examples aren't something I can really give out given that they're proprietary. this last winter I started a whole guest-kernel based syscall intermediation and distribution framework in rust just for the application. with all kinds of design documents. I was about 30% finished by the time I landed a job somewhere else :)

      but I still applaud the intent. I self-selected out by giving into scope creep

      • TZubiri 2 hours ago
        I'd be interested in the context if you'd be willing to share.

        It sounds from the outside like Oxide has an interview process that requires some low level engineering work to be delivered? Maybe I got that wrong.

        • convolvatron 2 hours ago
          no, they want a questionnaire, a coding sample, and an example of technical writing. there's a reasonable interpretation of that that doesn't involve writing a distributed unix.
          • TZubiri 2 hours ago
            Sounds approachable, and something that would be evaluated based on merit.

            As usual, I'm assuming the assignment is evaluated based on a reasonable time-commitment. From what the recruiting experts tell me, it's a good strategy to spend as much time as possible, the deliverable is better, and the optics aren't bad either, it signals investment into the application instead of signalling spray and pray application broadcasting.

    • Melatonic 1 hour ago
      Seriously - looks so cool
    • thraway3837 2 hours ago
      Just a gentle reminder that a company may portray itself as cool to customers, but is not cool to their own current or future employees.

      Their interview process was shady. There was a post here about 1-2 years ago that was a link to their interview process and how open and transparent they were. The post itself was from an employee and a fellow commenter who was gaslighting folks was also an employee. Several folks complained about the tremendous amount of homework they had to do after the initial screen, and once submitted, were ghosted. One of employees repeatedly rebutted that claim in the comments, and they did this for quite a few commenters. Was a not a good look. I doubt much has improved since then as seeing the comments below confirms the same mess.

      Don't spend time being amazed by folks who won't treat you right. It just ain't worth it.

      • bcantrill 1 hour ago
        I'm not sure what "mess" you're referring to -- that we have a writing-intensive hiring process? That we get a lot of applicants? That we therefore end up rejecting a bunch of people? That we read application materials thoroughly? That we don't provide specific feedback on individual applicants (even though we explicitly state that/why we don't)?

        To state clearly what I feel we have said many times: Yes, it's hard to get a job at Oxide. Yes, we get a lot applicants. Yes, we ask a lot of applicants upfront. But the payoff (and the reason it's worth the risk and the work for the right person!) is an extraordinary and uplifting team -- one that I daresay each of us counts as being of unparalleled breadth and depth in our careers.

        • contingencies 47 minutes ago
        • mrcwinn 48 minutes ago
          I understand that all employees have equal salary pay (apart from sales people who can earn more and are valued higher). Do all have equal equity and voting rights, at least within common stock?

          And since transparency is a core value and principle, will you commit to sharing your cap table publicly?

          • bcantrill 16 minutes ago
            I appreciate that our approach to compensation leaves some with overwhelming feelings of whataboutery, but no, we (of course?) do not have equal equity: as we have said (several times?) equity broadly compensates for risk -- and risk has gone down over time. (I used to tell people to "value the equity at zero"; I don't say that any longer because it plainly isn't.)

            In terms of the cap table: that's a bit of an odd request? On the one hand, there are no real secrets hanging out on our cap table -- but on the other, based on your tone, it doesn't feel like the request is terrible earnest? (And, I hasten to add, transparency is a value -- not a principle.[0])

            [0] https://rfd.shared.oxide.computer/rfd/0002

      • dijit 1 hour ago
        Devils advocate (really not affiliated with oxide, but I have worked for a “desirable” employer before).

        How would you handle a few thousand applicants for a single role?

        I think no matter what you do it will feel inhumane, we can argue that a few hours of work for a take home test is inhumane too, being ghosted after doing one definitely wouldn’t pass my personal bar of acceptability, but if its the first stage and the task would take a properly qualified applicant less than 30 minutes then I can’t fault.

        How would you do things? remember that it has to scale and you cant leave any gaps based on human fallibility (HR/Hiring Managers are humans and will forget if there are too many things going on at once).

        • jaggederest 1 hour ago
          There's a simple answer, if someone is doing a substantial amount of work for your interview process, pay them an amount of money that is more than zero but less than "do job interviews for a living". Or provide that amount times two to a charity of their choice.

          I've done this for hiring before, for people who reached the "put substantial effort in" stage (in my case basically 2nd or 3rd round work sample stuff), and it was a great way to make sure we got good signal and they felt respected.

          • dijit 1 hour ago
            “put substantial effort into it” is such a personal thing.

            DDG hires like this, actually, and if I recall correctly I would be paid a flat fee, it would take a week, and the work I did would be part of something genuine in DDG, maybe a bug or something.

            Now, that probably sounds good to you, but taking a week out of my current employment is not going to happen- there’s an incentive to go “over the hours” inherent to the ask, even if you’re paying me a flat rate, I might lose to someone equally qualified who puts in 1.01n into the task, so I should put 1.02n (etc; ad infinitum).

            Which is part of the issue with all take home assignments. I have given out take home assignments (given to HR to be administered) which should take a qualified candidate 20 minutes to finish beginning to end (as in, including syncing the project, setting up their editor, exploring the problem, googling around about things, trying it out and then following up with the email to HR). I don’t doubt for even a moment that someone has spent several hours on this problem- because they’re not qualified.

            Passing the HR barrier in that case will not help them unfortunately, because they’ll get to talk to me, and I will disqualify them in all likelihood, and candidates are told that it should take not more than a half hour, but en masse: people don’t listen.

            The trouble is, theres thousands of applicants, a handful of HR, and one me.

            Not to be on some kind of pedestal (I’m not), but the problem doesn’t scale, you need only apply the tiniest amount of systems thinking to see it.

            • jaggederest 1 hour ago
              Thousands of applicants reaching the substantial work stage is a failure of the systems thinking you're talking about. Hundreds of resumes nearly always gets narrowed down to perhaps a dozen or two at most at the screening stage.

              And I would make it very clear that putting in more than 30 minutes of work, timed, is a disqualifier, and I would sleep well at night clearing all those people out of the queue.

              • dijit 1 hour ago
                Hundreds of good applicants can’t be whittled down to a dozen without being very picky about things in the resume which may just be a poor representation.

                You will bias heavily along some kind of axis, preferred previous employers or location, age, etc.

                You add a lot of bias into the system by trying to further scrutinise otherwise meaningfully qualified people on paper.

                • jaggederest 56 minutes ago
                  Once again, you're misunderstanding the goal of the system if you think that it's necessary to deliberately whittle down hundreds of good applicants through careful process to get a great hire.

                  Hint: you don't even need to evaluate most candidates at all. Random sampling is sufficient and provably bias free.

                  • dijit 53 minutes ago
                    Reminds me of something I heard once.

                    > Whenever I get a stack of resumes, I throw half of them in the trash

                    > I sure don't want unlucky people on my team.

                    What do you send them as a response "sorry, we're going ahead with other applicants" - "you have not been selected this time" -- what happens if you start needing to dig through that pool of now rejected candidates?

                    Peak humanity.

                    • jaggederest 52 minutes ago
                      > what happens if you start needing to dig through that pool of now rejected candidates?

                      I acknowledge that I am reaching back out, and they may not be available.

                      Like a human does.

                      > Reminds me of something I heard once.

                      >> Whenever I get a stack of resumes, I throw half of them in the trash

                      >> I sure don't want unlucky people on my team.

                      I was actually about to make the same joke.

            • ThePowerOfFuet 4 minutes ago
              >should take a qualified candidate 20 minutes to finish beginning to end (as in, including syncing the project, setting up their editor, exploring the problem, googling around about things, trying it out and then following up with the email to HR

              So about six minutes for the problem itself, then?

          • IshKebab 1 hour ago
            Yeah I just got a new job and they sent me swag for getting to a certain (quite early) stage in the interview process. Awesome idea.

            It was for an investment bank though and they have essentially unlimited money. I can't imagine any of the other companies I've worked for would be remotely generous enough to do the same.

            • jaggederest 53 minutes ago
              Hiring is expensive linearly to the salary of the people you're trying to hire, so if any of the companies you've worked for were trying to hire well, it'd be a rounding error. Back of the envelope is 90 days of salary, minimum, is the cost to hire, so there's no reason to be miserly about it - if you can't afford it, you can't afford to hire at all.
          • fred_is_fred 1 hour ago
            From a legal and financial perspective it seems like it would be difficult to pay people to do interview homework. There's tax implications and other issues like state labor laws.
            • icedchai 1 hour ago
              People do contract / temporary / 1099 work all the time. It's very simple.
        • throwaway219450 1 hour ago
          If you truly believe you’re “scaling” you do it the Google way and have a strict loop with a good rubric for the interview so applicants are comparable. The whole point of that system is thousands of people and hundreds of interviewers, and a very standard process. I’ve always found it pretty fair even with some randomness in scoring.

          You shouldn’t be giving take homes unless they’re either short, or the applicant passed a screen and you’re investing time. Otherwise how are you “scaling” the review? Claude? Hidden test suite (not bad)? Some sort of leaderboard (bad, rewards people with time), something else?

          • dijit 1 hour ago
            I’ve been through the Google process and I wouldn’t consider it to be the opposite of inhumane.
            • throwaway219450 18 minutes ago
              Well “humane” and fair aren’t necessarily the same, and some people hate loops.

              I like programming problems, spending a day at Google was fun, they put me up in a fancy hotel, and the interviewers were nice. Like it was clear a lot of time and money had gone into the process (6-8 hours of dev time is not cheap), not a zoom and ghost like most companies.

            • DANmode 1 hour ago
              Use less negatives.
              • dijit 1 hour ago
                it was intentionally this way
                • DANmode 1 hour ago
                  I probably should have spotted that, lol
      • oytis 29 minutes ago
        With every interview process you say "yes" only once and "no" many times. Where there are a lot of candidates, then many more times, while spending less time on each candidate. There is no way to design a process that will not leave the majority of candidates disappointed - as soon as they are up front with the amount of work you'll need to do, it sounds ethical to me
      • marshray 1 hour ago
        I'm seeing the phrases "tremendous amount of homework", "substantial amount", and "few hours".

        Does anyone have an actual estimated time we can discuss?

  • piker 49 minutes ago
    Oxide hitting stride just in time for the memory crisis. I hope they can sustain because they have the coolest stuff, and the podcast is great.

    I guess the world of atoms is still hard enough that you can publish an interactive spec of your product and not have to worry about it being immediately copied.

  • andrewl-hn 2 hours ago
    This design feels very obvious-in-hindsight. Consolidate power adapters and networking, replace cabling with pluggable slots. It's something similar to what IBM mainframes or Sun cabinets could've looked like. Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this, even at their peak in 2000s or during cloud boom in 2010s. I wonder why?

    Beautiful machine, and fun to see Illumos heart still beating inside!

    • bri3d 2 hours ago
      > Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this, even at their peak in 2000s or during cloud boom in 2010s.

      Not so sure about this one. HCI (Hyperconverged) rack units (where storage and compute live in the same racked systems) and "blade servers" have been a thing for a really long time now; compute sleds aren't what's novel here.

      Rack-level DC conversion is also not particularly novel, although underutilized IMO. It was pretty popular in HPC style density applications for awhile (see HP/SGI Altix 4000 for a good old example).

      What's unique about Oxide is that they went all the way down to the firmware and then back up, rather than doing commodity hardware integration or reselling - for example, you can get something like a Supermicro EVO:Rail, but it will be running VMWare, not a fully integrated platform.

    • steveklabnik 1 hour ago
      The big difference that everyone is missing in this subthread is that Oxide is about the hardware and the software.

      There are systems which have similar overall hardware designs, but they are usually integrating a large amount of hardware and software from multiple vendors. Oxide is much closer to "everything is produced by Oxide."

      I wrote this back in 2022, and it's still fundamentally relevant today: https://news.ycombinator.com/item?id=30678324

    • baby_souffle 2 hours ago
      > Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this,

      Dell and HP both have "blades" that plugged into a blade-chassis. The chassis had all the lights out mgmt as well as power/networking integrated so the blade was basically a metal box with compute/memory/storage and it just slid in to the dock.

      I am sure that supermico had something like this as well

      • XorNot 6 minutes ago
        Blades basically died out is the thing - AFAIK no one really wanted them and honestly the same is a risk for what Oxide is doing too.

        Blades have the basic issue of "how often do you want an unpopulated chassis?" - answer, never.

        So really they're solving for replacing a failed piece of hardware.

        But how often do you need to do that, what's it worth to you? If it makes sense then the statistical window where it does is tiny.

        And if you own more then 1, like an entire rack, then do you even care? Because above some scale you're just going to wheel the rack out rather then go and pull individual units.

        Basically the scaling is against you: for a highly manageable bladey rack unit, you've got to be small enough that one server matters, large enough you need the swap out to be low labor, but not so large you could just wait for the rack to go down. And this has to be worth enough to justify the price premium and vendor lock in (because at rack scale you just buy a rack of the cheapest whatever from any vendor and make them compete on price - at one job bringing our computer management in house triggered an immediate 10% price drop because we threatened HP with using another supplier at all).

      • Melatonic 1 hour ago
        Cisco does too and theres another hardware virtualization layer below the normal ones ( so for example you can have many virtual nics per actual nic, etc)
        • casumel 33 minutes ago
          Cisco UCS! A great hardware platform, albeit quite expensive.
    • foobiekr 44 minutes ago
      Blade servers have been doing this for 25+ years.
    • wmf 2 hours ago
      The vendors did make blades in the 2000s.
      • creaturemachine 1 hour ago
        I thought surely this isn't just blade servers, that those compute shelves were full of GPUs or something novel, but no just blades reincarnated. I used to support HP's baby version of this, the c3000.
      • jeffbee 1 hour ago
        Also, a big cabinet into which you plug varying amounts of hardware capacity, then use the control plane to partition into various virtual resources, describes at least at the conceptual level IBM going back decades.
    • manwithopinions 2 hours ago
      I learned about blade servers back in ~2010 because Blizzard used to run World of Warcraft realms on them and auctioned them off for charity.

      https://warcraft.wiki.gg/wiki/Server_blade

      https://en.wikipedia.org/wiki/Blade_server

    • fred_is_fred 1 hour ago
      > Somehow hardware giants like Dell, HP, SuperMicro, etc didn't make a product like this, even at their peak in 2000s or during cloud boom in 2010s. I wonder why?

      They all did. HP had Super Dome and blades and Synergy. Dell had similar.

  • dcre 1 hour ago
    • piker 45 minutes ago
      Thanks for sharing that. Very cool - was this sales product that you decided to release or has it always been public?
      • benjaminleonard 33 minutes ago
        Sales/marketing. Though there's some ideas floating around on using it to document something like a CRU replacement.

        We need to get it cross-linked from the main site still.

        • piker 18 minutes ago
          Makes sense. I was looking for a link from the main site and noticed there wasn't one. A great leave-behind for sales. Nice job.
  • GL26 39 minutes ago
    Very instructive on how a computer rack works. I cought only later on that it was Oxide specific, but the design decisions seem so obvious to me that they look like industry standards. What about firmware architecture ? How do you design for reliability ? And how often does an infra like that stop working bc of a hardware problem ? And firmware ?
  • bradfa 1 hour ago
    Curious how robust the (what looks like PCIe edge connector slots) connection to the drives is in practice. Obviously converting from the horizontal mainboard to a vertical drive requires such a connection, making it a plug-in card at least allows for replacing the card if it breaks/wears/etc, and mounting the front of the adapter card to a bulkhead should prevent much shifting of the card in the slot. Neat design and reuse of a cheap high speed connector.
  • bradfa 1 hour ago
    How easy is it to swap the fan bar out for a failed fan? It looks like a single unit holding all the fans. Can the sled be pulled but retained in the rack and then fan bars removed and reinstalled without fully removing the sled and without tools?
    • sudomateo 1 hour ago
      The sleds blind mate to power and network on the backplane so removing the sled will disconnect it. Once the sled is disconnected you pull the fan assembly out and replace it.
  • girfan 1 hour ago
    Cool tour. I haven't kept up with their developments; what kind of workloads have they been pushing for? Since they don't seem to have any specialized accelerators in the Compute Sled, I am assuming they are not targeting AI workloads for now?
    • connicpu 31 minutes ago
      Hard to compete there when most of the margin is going to NVIDIA, pretty much impossible if you don't have the scale of Dell/Supermicro/etc
      • adastra22 6 minutes ago
        The Oxide approach, I think, would be to pick an open interconnect standard and work with one or more NVIDIA competitors to make hardware for it. E.g. AMD and a few of the specialized AI hardware companies. Essentially provide an open TPU standard.
  • dralley 2 hours ago
    It's a shame they missed out on the AI server boom.
    • panick21_ 56 minutes ago
      They didn't. They actually said they have huge demand because of AI companies. Turns out, when the AI is surfing the web or using doing other task, that's a CPU. So they also have demand for CPU.
  • arjie 2 hours ago
    Oh wow, they're on the Epyc 9005 series now. Very cool. Dude, what a monster of a machine that rack is haha. Bloody hell.
  • drudolph914 1 hour ago
    big fan of oxide, and love the demo overall, but at the same time, I can't shake the feeling that these kinds of 3D demos are a bit gimmicky/cheap nowadays. pages like this used to be a signal for a high-end product and or a benchmark for good engineering. now though, we all know that this kind of work can be vibe-coded with threejs fairly quickly if you have the assets. idk ... it feels like it's trying to capture my attention through flashing lights instead of letting the work stand on its own

    I don't want to subtract from the demo too much, b/c I do love oxide, but I do see this as a trend that more people will use to garner attention until it's too overdone - at which point, 3D will revert to being used for more practical use-cases

    EDIT: typos

    • benjaminleonard 34 minutes ago
      IIRC the CAD exports were in the many millions of polygons, so there's substantial work in preparing the assets for the explorer. Kenan (the 3d artist I worked with) did fantastic work cleaning and optimising them. I'm somewhat jealous of hardware startups with products without quite so many parts!

      It's part of the reason I'd waited so long before making this, I knew it was going to be a lot of work. There's parts that Claude was especially useful for, like perf testing, debugging and animation. But the first half of the project was done almost entirely by hand.

      • drudolph914 6 minutes ago
        I appreciate ya commenting, and let me tell ya directly that the work is great! I'm an old-head and curmudgeon - my thoughts here are about the world at large. your work here is great, please keep it up!
    • mnsc 1 hour ago
      My feeling? I ask myself the same question as I do for for everything nowadays. Yeah this might be ai slop, but is it good ai slop? If it's good, like this, I feel good. If it's bad, I feel bad.

      Human effort as a proxy for quality... that ship has sailed. And that makes me feel frustrated.

      • drudolph914 34 minutes ago
        yeah, I agree with you. I guess the part that feels disingenuous is that these kinds of projects are some-what cashing in on attention from people who still who see threejs slop as "high-effort UIs." this speaks more about me than other ppl, I'm just personally already at the point where all of this feels closer to slop rather than quality. I guess as this trend plays out, we'll all eventually feel somewhat similar on the yet-another-3d-UI-slop trend
  • Kostic 1 hour ago
    A very beautiful website and a machine. Oxide folks should be proud, you can see the love that went into it.
  • dorongrinstein 2 hours ago
    I met Bryan Cantrill the CEO of Oxide many years ago. He's awesome. I am rooting for Oxide to become as big as Dell.
  • PoignardAzur 2 hours ago
    The screen goes black for me after ~5s. I'm using Firefox on Linux, probably something to do with that.
  • kevinrineer 1 hour ago
    I'm a bit surprised that this kind of incredibly engineered hyperconverged rack system isn't doing some form of liquid cooling.
    • wmf 1 hour ago
      99% of data centers do not have liquid available to the rack. Liquid within the rack doesn't really help.
    • schainks 1 hour ago
      Not worth the capex to do that. Air cooling with evaporators up front is pretty standard and works great, not to mention fewer moving parts.
  • burnrate 3 hours ago
    I wonder how much a rack like this costs
    • LoganDark 3 hours ago
      IIRC I heard that a single rack is upwards of $600k–$1m, but that was before the AI boom/crisis.
      • TZubiri 2 hours ago
        I don't recall if the price was confidential, but I will say that it's higher than that, and that it has been influenced by the RAM cost increase.

        If I recall when comparing to competition, it was premium priced, for sure, but it's more that it's so dense that you had to compare 1 Oxide rack to like 4 commodity racks. Spec for spec I recall that the premium for the verticality wasn't that high.

    • jeffbee 3 hours ago
      With only 288 128GiB DDR5 ECC DIMMs? I'm sure that doesn't cost much.
  • joshAg 2 hours ago
    may the Sun never set
  • DiabloD3 3 hours ago
    Thats neat.
  • RobLach 3 hours ago
    Neat