The updated syntax looks brilliant — can't wait to try it out by upgrading some of my older notebooks! I was worried about the future of Observable since the Canvas announcement — it felt like notebooks were being slowly phased out, but I'm glad to be proven wrong.
Any timeline on bringing the new format to the Web editor? For me, the biggest draw of Observable is being able to instantly start exploring and visualizing some data without ANY setup (creating some local files, installing dependencies, launching a dev server etc.) — just click "New notebook" and off you go, frictionless. I hope this remains a priority.
Also worth mentioning the data viz community aspect, which I used to enjoy a ton — it's a bummer that things like being able to follow certain users and see their new notebooks in a feed were phased out, and that trending notebooks are now hidden under the Help menu instead of being featured on the user's homepage. This might be one of the reasons I stopped coming back to the platform as often, although I realize building a community is very difficult and hard to monetize — just thought I'd share.
Thank you. I don’t want to commit to a public timeline on the web editor yet, but rest assured it is a top priority, and we also value the convenience of a web editor, instant collaboration, and community!
I'm not sure I understand what the offering is here. I read the website, and watched the video, but it doesn't really make sense.
Is Notebook 2.0 just a HTML subset that starts with e.g. <notebook> instead of <html>?
Then what's the notebook kit npm package?
Then there's an editor?
I've come away from the page a little irritated, because starts with hyperbole like "the future of notebooks is here", and then the "hello world" in the new notebook 2.0 format looks like HTML, and a video of what could have just been a generic AI chat window.
Maybe I'm too hungry/tired to make sense of it right now. Will be interesting to read what others think.
They probably could have explained better what hasn’t changed. Observable Notebooks work like a spreadsheet or makefile. The cells aren’t executed top to bottom (like a Jupyter Notebook). Instead they’re executed in dependency order.
Looks like this is still true with the new system:
> The Build API analyzes cell source code to find unbound references, such that the appropriate variable graph can be initialized using the Observable Runtime. For non-JavaScript cells (such as Markdown, HTML, and SQL), the Build API also transpiles the source into a tagged template literal expression.
This is unlike other document formats. They made the cells look like standard JavaScript now (versus the JavaScript variant they had before) but the file format is necessarily different, because the execution model is different. There is a build step to ensure that the cells will be executed in the right order on page load.
The video demonstrating the AI powered desktop app perfectly demonstrates my discomfort with AI. When asked to fix the mapping between county codes, it generated a lookup table itself.
HAVE FUN TESTING THAT. And when you don't, enjoy random countries being mixed up but it mostly working.
Yeah. I can believe that this will improve in the future, and this is a technology preview and all. But my takeaway from the video is that it is still too vibey for me to trust it. If I was coaching a junior data analyst at my company, and they wrote this code, I would happily give some feedback so that they can improve. With AI, I'd probably just do the analysis myself.
In addition to the hardcoded lookup table, here are some other notes on the generated code:
1. Silently assuming that the first page of results contains all of the data feels a bit fragile. If we're not going to bother with paging otherwise, I'd at least assert that the first page contains everything.
2. Unlike the code comment claims, 2022 is not the latest year that has data available, 2023 is. The reason this is worrisome is not that the difference is massive, but because of methodological implications. It looks like the year 2022 came from "remembering" details from some other code that was analyzing this same data set instead of just looking at current version of the data set we're supposed to be analyzing.
3. The code for removing the world aggregate doesn't actually work (although it doesn't matter for the map). The place where it says d.country.id !== "WLD" should be either d.country.id !== "1W" or d.countryiso3code !== "WLD" instead. Also, if it would actually be important to filter this then presumably it would also be important to filter out a bunch of other aggregates as well.
4. The text says "darker colors indicating higher life expectancy", which is pretty much the opposite of how I would describe this color scheme.
5. The analysis given is: "Notice how certain regions tend to cluster together in terms of life expectancy, reflecting similar economic, healthcare, and social conditions". This is such a bland take. It feels like something I could have written before seeing the data. I would try to encourage the analyst to look for something more concrete and more interesting.
6. The actual thing that immediately pops out from the map is that the Central African Republic is a massive outlier with a life expectancy of 18.8 years, whereas every other country is over 50. This doesn't seem plausible. I would do something about that just so that it doesn't mess up the colors of the entire map.
I'd love to love them but I don't: I can't seem to justify moving away from Python to use this. What advantages does JS offer for this use case? I' ve never felt that I couldn't do any visualization with Python (but I don't do nice newspaper figures).
It depends where you're coming from, I suppose. The web is pretty popular. Web developers are more familiar with HTML, CSS, and JavaScript, and with various visualization libraries in the JavaScript ecosystem. I quite like Observable Plot [1].
I'm not familiar enough with Python or Jupyter to know how you would build similar visualizations with them. What would you use?
Im in the same boat. In theory JavaScript holds more potential to make finely crafted visuals. But you're right, the ecosystem is so mature I still find other ecosystems lacking.
But if anything would change my opinion this has the right set of values
I'm not really a JavaScript person either, but that hasn't been the worst problem with observable. Tools like vega-lite and importable tools from other notebooks really reduces the amount of code I need to write. Like, I'm not writing d3 code for hours for a simple choropleth. What makes it easier is that I can query objects using SQL rather than lengthy JavaScript code.
Exporting as client-side webpages, maybe? I liked using it to prototype some interactive D3.js charts that I would later move to my website. Also some people just are more comfortable with JS than Python
I used the older online notebooks for a bit and really loved the immediacy but I’m always hurting for TypeScript syntax support. The is this a thing w 2.0? Something I’ll need to add later with Vite and only available locally?
It was a bunmer to not be able to copy paste code from my projects into observable. With local notebooks I see more potential for homegrown workflow but for me the benefit was always the platform and how starting a new thing was as simple as going to a webpage and clicking plus.
Anyways excited to play with this once it ends up in the online editor
We’re interested in adding a TypeScript cell mode, probably first with type stripping rather than validating types. I was hoping for a pure JavaScript implementation; maybe we’ll try https://github.com/bloomberg/ts-blank-space? I’ve also been looking at https://github.com/val-town/codemirror-ts for the editor, but it’s a lot of machinery.
The Observable Desktop macOS app is a 11.8MB download which decompresses to a 23.9MB application. In a world filled with 200+MB Electron apps this is a very refreshing change!
I wish Bostock et al got paid. This is literally the pain of doing open source for some. You basically watch as other people get rich off of your life’s work.
That rambling aside, I like the idea of having a desktop app. Not sure where this notebook fits in with all their other products. They have the Framework which is actually pretty useful.
I've found Bostock Observable to be a fascinating and inspiring platform since the outset in my limited experiments with it. I wish they'd give it a name that isn't a well-known design pattern that it implements, because I end up having to call it "you know, Bostock's thing" in conversation.
However, I definitely don't want to have to type "<script id="2" type="module" pinned>" in order to calculate 1 + 2, and I don't want to have to scan past that in order to find the actual calculations. So I guess I'll have to keep being inspired by Bostock Observable Notebooks 1.0, which I can keep writing in GitHub-flavored Markdown, because it's free software and already checked into Git.
I work on a quantitative user research team, and we've found notebooks to be really useful for crunching and visualizing the data from experiments we've run.
I work at Google and my UXR colleagues are more comfortable with Python than JS, so we use Google Colab, but I'd use Observable if those weren't true.
Out of curiosity, why `<notebook>` as an element and not a standard web-component `<observable-notebook>` or something like that if the goal is to go all-in on "normal" web development?
They made the syntax look familiar so that standard editors (and LLM’s) will work, but this is still a custom file format that requires a build step to convert into an HTML page. See my other comment.
The build tool is open source and unlike a Jupyter notebook, it’s easy to edit without a custom editor. So you don’t need their editor, but it will still be nicer to edit with their editor.
self-plug that you might be interested in: I built a thin wrapper around the Observable Runtime that allows you to more easily use it in a pure HTML context: https://maxbo.me/celine/
I think the goal is explicitly not to go all-in on normal web development, to maintain some opportunity to monetize.
D3.js started as/still is just a (collection of) JS libraries. Then Observable came along, and while it is a nice tool for tinkering with D3, it was not at all obvious how to then move your finished Observable D3 viz to a fully self-hosted, regular website (without paying a subscription to use their custom runtime or servers or whatever).
Now I guess they realize they strayed way too far from "the web" and are back-tracking.
This is really, really incredible stuff. I can't wait to replace a bunch of terrible BI dashboards.
I've seen and used so many amazing in-house dashboarding and reporting tools, and this suite blows them all out of the water. Can't believe this is all open source too.
I think this is a great way to slice up the product/feature set from a monetization perspective: hosting w/ collaboration is paid, but the tools are all free to use.
This looks great. I love the idea behind notebooks and for a long time it was my favourite environment to program in. But slowly I stopped using them because it never quite felt like the code was entirely mine, and alternatives became easier due to llms. This looks like exactly the remedy I was hoping for. I'm excited to start using them again.
I'd love to see support for inline TeX in the markdown. Back when I used observable on the web, I hacked up a `md2` string template to replace /\$(.*?)\$/g with the katex equivalent before calling `md`, but I don't know if there is a workaround in the desktop notebook application.
Not exactly. Jupyter notebooks execute top-to-bottom at startup and then you can manually execute cells in whatever order you choose after that. I believe it’s the manual execution that gets people, because it can have inconsistent results. But just rerunning the entire file automatically (like reloading a web page) might be slow if some build steps are slow (they can read or write huge files), and some people have slow steps in their notebooks. They could use manual cell regeneration to intentionally avoid rebuilding cells, when stale data is acceptable.
On the other hand, I don’t know if Observable Notebooks 2.0 even does incremental builds anymore? If not, it’s just about presentation order, similar to literate programming.
Observable Framework had data loaders for creating static snapshots of data at build time. I don’t see any mention of how to handle data loading in Observable Notebooks 2.0.
Because it makes the reactive model work - and work quite efficiently. Cells don't re-run just because of edits, but also if the reader interacts with any of the interactive inputs.
Why should irrelevant nerd junk be at the top of my notebook? If the point of the notebook is to inform and educate the reader, that's a distracting limitation.
They have or at least used to make their notebook format public, and I always thought if I ever see someone else adopt it then maybe the project will be worth looking at again. I think there could be better ways to capitalize on d3 but this one seems self limiting.
It's pretty clear the future is not really here yet when the LLM can't even debug a simple table join issue on its own. The code to produce the table and chart should reasonably be like 2 lines of code with proper tool design. It looks like the LLM accomplishes it in 100 LOC and still can't manage to do it without significant human intervention.
And this is the take they published to sell the thing!
Observable's pricing is bit out of bounds for what they offer. It feels like a lot of these data startups are just looking for large companies that will unsuspectingly spend a ton of money on something they don't need.
Any timeline on bringing the new format to the Web editor? For me, the biggest draw of Observable is being able to instantly start exploring and visualizing some data without ANY setup (creating some local files, installing dependencies, launching a dev server etc.) — just click "New notebook" and off you go, frictionless. I hope this remains a priority.
Also worth mentioning the data viz community aspect, which I used to enjoy a ton — it's a bummer that things like being able to follow certain users and see their new notebooks in a feed were phased out, and that trending notebooks are now hidden under the Help menu instead of being featured on the user's homepage. This might be one of the reasons I stopped coming back to the platform as often, although I realize building a community is very difficult and hard to monetize — just thought I'd share.
Is Notebook 2.0 just a HTML subset that starts with e.g. <notebook> instead of <html>?
Then what's the notebook kit npm package?
Then there's an editor?
I've come away from the page a little irritated, because starts with hyperbole like "the future of notebooks is here", and then the "hello world" in the new notebook 2.0 format looks like HTML, and a video of what could have just been a generic AI chat window.
Maybe I'm too hungry/tired to make sense of it right now. Will be interesting to read what others think.
Looks like this is still true with the new system:
> The Build API analyzes cell source code to find unbound references, such that the appropriate variable graph can be initialized using the Observable Runtime. For non-JavaScript cells (such as Markdown, HTML, and SQL), the Build API also transpiles the source into a tagged template literal expression.
This is unlike other document formats. They made the cells look like standard JavaScript now (versus the JavaScript variant they had before) but the file format is necessarily different, because the execution model is different. There is a build step to ensure that the cells will be executed in the right order on page load.
It's a neat tool that solves a lot of specific sharing/visualization problems (think of newsroom data analysis).
I like observable but I hate how much it feels like the service has been taken over by product managers looking for that next bump.
HAVE FUN TESTING THAT. And when you don't, enjoy random countries being mixed up but it mostly working.
In addition to the hardcoded lookup table, here are some other notes on the generated code:
1. Silently assuming that the first page of results contains all of the data feels a bit fragile. If we're not going to bother with paging otherwise, I'd at least assert that the first page contains everything.
2. Unlike the code comment claims, 2022 is not the latest year that has data available, 2023 is. The reason this is worrisome is not that the difference is massive, but because of methodological implications. It looks like the year 2022 came from "remembering" details from some other code that was analyzing this same data set instead of just looking at current version of the data set we're supposed to be analyzing.
3. The code for removing the world aggregate doesn't actually work (although it doesn't matter for the map). The place where it says d.country.id !== "WLD" should be either d.country.id !== "1W" or d.countryiso3code !== "WLD" instead. Also, if it would actually be important to filter this then presumably it would also be important to filter out a bunch of other aggregates as well.
4. The text says "darker colors indicating higher life expectancy", which is pretty much the opposite of how I would describe this color scheme.
5. The analysis given is: "Notice how certain regions tend to cluster together in terms of life expectancy, reflecting similar economic, healthcare, and social conditions". This is such a bland take. It feels like something I could have written before seeing the data. I would try to encourage the analyst to look for something more concrete and more interesting.
6. The actual thing that immediately pops out from the map is that the Central African Republic is a massive outlier with a life expectancy of 18.8 years, whereas every other country is over 50. This doesn't seem plausible. I would do something about that just so that it doesn't mess up the colors of the entire map.
Yes but, probably what they are shooting for are mini apps that you can just copy pasted into a front-end codebase and not some weird-ass python dsl
I'm not familiar enough with Python or Jupyter to know how you would build similar visualizations with them. What would you use?
[1] https://observablehq.com/@observablehq/plot-gallery
But if anything would change my opinion this has the right set of values
It was a bunmer to not be able to copy paste code from my projects into observable. With local notebooks I see more potential for homegrown workflow but for me the benefit was always the platform and how starting a new thing was as simple as going to a webpage and clicking plus.
Anyways excited to play with this once it ends up in the online editor
That rambling aside, I like the idea of having a desktop app. Not sure where this notebook fits in with all their other products. They have the Framework which is actually pretty useful.
They do get paid. Observable sells team and enterprise licenses: https://observablehq.com/pricing
However, I definitely don't want to have to type "<script id="2" type="module" pinned>" in order to calculate 1 + 2, and I don't want to have to scan past that in order to find the actual calculations. So I guess I'll have to keep being inspired by Bostock Observable Notebooks 1.0, which I can keep writing in GitHub-flavored Markdown, because it's free software and already checked into Git.
I work at Google and my UXR colleagues are more comfortable with Python than JS, so we use Google Colab, but I'd use Observable if those weren't true.
The build tool is open source and unlike a Jupyter notebook, it’s easy to edit without a custom editor. So you don’t need their editor, but it will still be nicer to edit with their editor.
FWIW, the HTML custom elements spec does require a hyphen: https://html.spec.whatwg.org/multipage/custom-elements.html#...
D3.js started as/still is just a (collection of) JS libraries. Then Observable came along, and while it is a nice tool for tinkering with D3, it was not at all obvious how to then move your finished Observable D3 viz to a fully self-hosted, regular website (without paying a subscription to use their custom runtime or servers or whatever).
Now I guess they realize they strayed way too far from "the web" and are back-tracking.
I've seen and used so many amazing in-house dashboarding and reporting tools, and this suite blows them all out of the water. Can't believe this is all open source too.
I think this is a great way to slice up the product/feature set from a monetization perspective: hosting w/ collaboration is paid, but the tools are all free to use.
For the time being I hacked something using Observable Framework [1], but still, it isn't the full set
1: https://observablehq.com/framework/
Edit: nvm just saw the notebook kit
No? How would they make money?
These data viz are great dashboard builders but need a local/offline solution for my needs.
On the other hand, I don’t know if Observable Notebooks 2.0 even does incremental builds anymore? If not, it’s just about presentation order, similar to literate programming.
Observable Framework had data loaders for creating static snapshots of data at build time. I don’t see any mention of how to handle data loading in Observable Notebooks 2.0.
And this is the take they published to sell the thing!