It helps to gain a 30,000 ft view before zooming into all the details:
Family Spatial symmetry Example
------ ---------------- -------
Bessel Cylindrical (r,θ,z) circular drum, heatpipes, waveguides
Legendre (Ordinary) Axisymmetric Spherical (θ) earth's bulge spinning on its axis
Associated Legendre Full 3D Angular Spherical (θ,ϕ) spherical harmonics
Laguerre Radial Spherical (r) radial pdf of electron in hydrogen atom
There’s a kind of grief that I feel when I think about how, over the last 70 years, engineers have tended to increasingly rely on tech that makes the math knowledge unnecessary.
The beauty of math has played a huge role in the career choices that I’ve made so far. I changed fields at one point because I realized that the math that practitioners used was almost exclusively all baked into CAD tools. I figured that if that was where the applied math lived, then I wanted to write software for a living.
With a lot of the AI tools now (which I use and appreciate daily FWIW), I’m beginning to see that even in math-heavy sub disciplines of software engineering (e.g. robotics), we’re not exempt from this trend.
This comment reminds me: I was in a used bookstore a few weeks ago with my dad (a retired engineer) and he bought up a bunch of the old math and engineering books because, he said, they approach their subjects differently and explain how things work, they walk you through the math and work to build up a detailed mental model of what's actually happening. His feeling was that modern textbooks make everything so abstract that the subject begins to feel divorced from what you're trying to do or understand, which makes getting an intuitive grasp on a topic really difficult.
He showed me some of the books from the 40's and 50's geared towards teens and honestly they did a better job of explaining things than the college textbooks I had to use -- they didn't go into fine detail, but they gave you a good framework to build on and a way of thinking about something which I feel is lacking from contemporary texts.
Tangentially, Feynman said something in a similar vein about the transition from analogue to digital. Kids couldn't tinker anymore and had no real impetus to learn some of the laws which underpin their workings, and most importantly, had no way to build an intuition around these laws through casual experimentation (i.e. fixing a broken radio with a handful of parts from Radio Shack).
Abstraction can be a very powerful tool, but growing up and learning in an environment which has these abstractions already built-in is a real handicap to building a deep understanding of a subject. You become dependent on the tools themselves instead of developing the knowledge used to build the tools -- in effect, the tools become limitations. Look at kids who grew up with iPhones instead of Amigas, or people already becoming dependent on LLM's.
There's a book called "Quick Calculus" (originally published in 1965, though I have the 1991 second edition) that my high school physics teacher gave me as a way to prep for physics class over the summer. It's a self teaching guide to basic calculus and is written to be used standalone without a lecture or grader. It has choose your own adventure stuff where if you get a question right you can skip over an extra explanation/example.
I've kept it around and used it for self teaching when I went down the rabbit hole of prepping for actuarial exams a year back, and again for prepping for going back to college to change careers to medical physics (yes, I have been looking at a bunch of different options to leave software).
I find it so much more elegant and straightforward than modern textbooks, and it's so much easier to focus on pen and paper vs screens. I've had people suggest Khan Academy and 3Blue1Brown, but I find I learn more deeply and retain better from a good old school textbook.
That's Kleppner & Ramsey's book, who shared the physics Nobel for the invention of the hydrogen maser. Ramsey had already come up with the key principle that made atomic clocks practical. (Edit: Kleppner was the co-inventor but didn't get a ticket to Stockholm, and in fact Ramsey was awarded for his other work on atomic clocks.) They were some of the all-time great experimental physicists. I didn't learn calculus from them but I have that book on the shelf.
"Programmed instruction" books like that one were big in the post-Sputnik era, part of a larger push to systematize STEM education. I've always wondered why it never caught on. I guess it's not ideal for a classroom setting where everybody gets dragged along at the same predetermined rate.
There are a few great books aimed at self study that I've used over the years, and this and the other one I really liked were both from the 50s/60s (Wheelock's Latin being the other). There's something about the seriousness while also not expecting you to be using any other materials or have the support of a lecture/grader that both of these books had that more recent books don't seem to replicate.
The For Dummies and other such lines of books for interested amateurs stay very surface level, while textbooks aimed at classrooms tend to assume that you have lectures, a grader, and, at this point, a login to an expensive portal. I do find higher level CS and math texts to be better quality, but I suppose that is because I am familiar enough with those fields to access them.
There's something about the assumption that the reader is a serious self motivated learner that authors in the 50s and 60s have that doesn't seem to be as common anymore.
I completely agree that so much interesting math is hidden behind hyper-proprietary tools that engineers are expected to use.
As you note, all the Finite Element tricks are hidden, all the meshing tricks are hidden, and while you can find some in old research papers, in general the main companies hoard them.
The landscape has been 3 / 4 forces:
1) individual academics that do interesting work but more often than not abandon the codes they produce after a paper or three.
2) Proprietary companies that hire enough expertise to take academic research, generalize it, and sell a hyper-proprietary product with very little competition because the expertise is hard to come by
3) The national labs which do amazing work, that release a body of reliable open source work, though some of it is tailored to their systems - usually hard to use / very few people know about it outside academia.
4) Open projects like OpenFOAM / OpenCascade that struggle along at the edges to create a cohesive whole that can compete with the big companies.
I think that LLMs offer an opportunity free us from this status quo:
1) The open source work of the past is much easier to use / decipher with an LLM
2) The research is there, it just hasn't been productized (through an open source vector) / made accessible yet
3) I do believe that LLMs make learning the past codes / getting ideas for new codes / extending codes easier
I think engineering tools are up for a shakeup, as both LLMs + new paradigms are emerging. There is really interesting research into new meshing methods / CAD representation (Brep vs SDF) / topology optimization.
I believe this is an opportunity for an open source ecosystem to start making inroads into the bottom layers of the engineering stack (CAD kernels / representation), and from there make inroads into meshing + make open source FEA more accessible.
I am excited by what the future might bring, and remain cautiously optimistic that just like no one buys a programming language anymore, the lowest levels of the engineering stack might become open source so that research is accelerated, engineers are freed from total lockin, and in general the ecosystem is much healthier and innovative.
To a degree, yes. That's not _that_ long after Einstein had to seek out math experts because he didn't know the theory behind 5x5 matrices. Math professors I know who started much later than that didn't really get how linear algebra was useful when teaching ODEs. It's not that it wasn't useful, but it wasn't commonly used.
I once had a mentor that was trying to help me write a CPU characterization and being a phd in math every suggestion he proposed involved derivatives. I was like, why would I describe I486 cache misses in terms of differential equations? To this day I still don’t know what he meant.
I think one could down the hardware to gates, then convert that to a partial difference equation (time discrete). In F_2 sum is "xor" and product is "and". "and" is bilinear. But it is not a universal gate. Don't shoot the pianist, but gpt suggestion is passing from linear to affine algebra. There traslating by 1 (traslation being summing, xor) is "not" and is an affine map, just as "and" is affine in each argument because being linear. Nice thing is that affine maps can be encoded as linear ones one dimension higher so "nand" can be seen as bilinear form acting on two 2D args (homogeneous bit coordinates in F^2*F^2), that is a twice covariant tensor. That tensor can be expressed less efficientely as a twice covariant, once contravariant one. The advantage of that being that you can map a circuit of "nands" to a tensor network of same topology so a big linear map L, so the difference operator becomes L-Id and that leads to the difference equation. mathse speaks confusingly both of difference and differential equations. Take this as a napkin reconstruction idea, not an account of what Feynman actually did (note also the above assumes RAM as part of the state of the CPU as "cache"/"mega-register-bank" and no I/O so model passes from Mealy machine to a fixed iterative transition function acting on initialized internal memory).
TBH, this might be a good time to figure out what he was talking about! A good continuous approximation of a discrete problem can radically simplify the situation. And current LLMs are extremely patient tutors.
> A good continuous approximation of a discrete problem can radically simplify the situation.
The question of discrete/continuous modelling has been on my mind, off and on, for years. How either approach can be more useful in certain contexts, or used to complement each other's strengths.
A recent example, I read about "real computation": hypothetical computing machines using infinite-precision real numbers. Around the same time I was reading articles on ultrafinitism and how "real numbers are not real", that spacetime itself may have emerged as discrete combinatorial structures.
The beauty of math has played a huge role in the career choices that I’ve made so far. I changed fields at one point because I realized that the math that practitioners used was almost exclusively all baked into CAD tools. I figured that if that was where the applied math lived, then I wanted to write software for a living.
With a lot of the AI tools now (which I use and appreciate daily FWIW), I’m beginning to see that even in math-heavy sub disciplines of software engineering (e.g. robotics), we’re not exempt from this trend.
He showed me some of the books from the 40's and 50's geared towards teens and honestly they did a better job of explaining things than the college textbooks I had to use -- they didn't go into fine detail, but they gave you a good framework to build on and a way of thinking about something which I feel is lacking from contemporary texts.
Tangentially, Feynman said something in a similar vein about the transition from analogue to digital. Kids couldn't tinker anymore and had no real impetus to learn some of the laws which underpin their workings, and most importantly, had no way to build an intuition around these laws through casual experimentation (i.e. fixing a broken radio with a handful of parts from Radio Shack).
Abstraction can be a very powerful tool, but growing up and learning in an environment which has these abstractions already built-in is a real handicap to building a deep understanding of a subject. You become dependent on the tools themselves instead of developing the knowledge used to build the tools -- in effect, the tools become limitations. Look at kids who grew up with iPhones instead of Amigas, or people already becoming dependent on LLM's.
I've kept it around and used it for self teaching when I went down the rabbit hole of prepping for actuarial exams a year back, and again for prepping for going back to college to change careers to medical physics (yes, I have been looking at a bunch of different options to leave software).
I find it so much more elegant and straightforward than modern textbooks, and it's so much easier to focus on pen and paper vs screens. I've had people suggest Khan Academy and 3Blue1Brown, but I find I learn more deeply and retain better from a good old school textbook.
https://archive.org/details/quickcalculussel00klep
I hadn't known about this one, although the Classical Mechanics text was great.
https://en.wikipedia.org/wiki/An_Introduction_to_Mechanics
And I learn from Wikipedia that Kleppner passed away just last year.
> It has choose your own adventure stuff where if you get a question right you can skip over an extra explanation/example
Reminds me somewhat of a book I made like this awhile ago, though mine is much more extreme and more for recreation I suppose. Might interest you: https://jtaylor.xyz/self-directed-guide-to-mathematics.pdf
"Programmed instruction" books like that one were big in the post-Sputnik era, part of a larger push to systematize STEM education. I've always wondered why it never caught on. I guess it's not ideal for a classroom setting where everybody gets dragged along at the same predetermined rate.
The For Dummies and other such lines of books for interested amateurs stay very surface level, while textbooks aimed at classrooms tend to assume that you have lectures, a grader, and, at this point, a login to an expensive portal. I do find higher level CS and math texts to be better quality, but I suppose that is because I am familiar enough with those fields to access them.
There's something about the assumption that the reader is a serious self motivated learner that authors in the 50s and 60s have that doesn't seem to be as common anymore.
As you note, all the Finite Element tricks are hidden, all the meshing tricks are hidden, and while you can find some in old research papers, in general the main companies hoard them.
The landscape has been 3 / 4 forces:
1) individual academics that do interesting work but more often than not abandon the codes they produce after a paper or three.
2) Proprietary companies that hire enough expertise to take academic research, generalize it, and sell a hyper-proprietary product with very little competition because the expertise is hard to come by
3) The national labs which do amazing work, that release a body of reliable open source work, though some of it is tailored to their systems - usually hard to use / very few people know about it outside academia.
4) Open projects like OpenFOAM / OpenCascade that struggle along at the edges to create a cohesive whole that can compete with the big companies.
I think that LLMs offer an opportunity free us from this status quo:
1) The open source work of the past is much easier to use / decipher with an LLM
2) The research is there, it just hasn't been productized (through an open source vector) / made accessible yet
3) I do believe that LLMs make learning the past codes / getting ideas for new codes / extending codes easier
I think engineering tools are up for a shakeup, as both LLMs + new paradigms are emerging. There is really interesting research into new meshing methods / CAD representation (Brep vs SDF) / topology optimization.
I believe this is an opportunity for an open source ecosystem to start making inroads into the bottom layers of the engineering stack (CAD kernels / representation), and from there make inroads into meshing + make open source FEA more accessible.
I am excited by what the future might bring, and remain cautiously optimistic that just like no one buys a programming language anymore, the lowest levels of the engineering stack might become open source so that research is accelerated, engineers are freed from total lockin, and in general the ecosystem is much healthier and innovative.
See https://math.stackexchange.com/questions/4752749/how-could-f...
The question of discrete/continuous modelling has been on my mind, off and on, for years. How either approach can be more useful in certain contexts, or used to complement each other's strengths.
A recent example, I read about "real computation": hypothetical computing machines using infinite-precision real numbers. Around the same time I was reading articles on ultrafinitism and how "real numbers are not real", that spacetime itself may have emerged as discrete combinatorial structures.
Reference:
https://althouse.blogspot.com/2024/12/during-cold-war-we-cla...