The code makes me smile, because it's nasty. This isn't like C4, a tiny but complete C compiler which does error checking on its subset. Instead, this is worse than Sector C, which takes every shortcut and just plain assumes everything in the source is right.
This "Python" just plain assumes for keywords:
Any "f" is a "for [x] in range[y]" (exactly that, no other for's).
Any "w" is a "while".
Any "i" is an "if".
Any "d" is a "def".
Any "p" is a "print("
Nasty, nasty.
(Also nasty is that the code snippets in the article has more comments than the github copy of the "readable" version. You need the article to understand what's going on.)
This is a just a bit too simple for a "Tiny Python". If somebody is willing to allow a few more K's of bytes, I'd love to see at least lists & dicts here--Lisp can do them!
For those who actually need something like this in production, there is Snek: <https://sneklang.org/> “Snek is a tiny embeddable language targeting processors with only a few kB of flash and ram.”
Yes, but compiling or modifying Snek from source is very challenging. I wish it was one single C file for an example base like Posix, instead of many files for many platforms plus a custom parser in Python (Lola).
This is really cool! It's so fun to see what you can achieve and what's optional. I have seen the 'single character variable' limitation in some other minilangs before, but using the source itself as the target of function calls and loops is new to me. It does make a lot of sense but I wouldn't have thought of that.
Yeah the amount of Python code that would work here is probably not a lot more than this specific FizzBuzz example. Lots of shortcuts taken, which I guess is understandable.
One could imagine an even smaller subset interpreter. It's an interpreter for a subset of Python, consisting only of the programs that print "Hello World". Since it doesn't do any error checking, for all other programs the output is undefined. Implementing it is very simple: Just ignore the input file, and print "Hello World". As a bonus, it's an interpreter for the subset of "Hello World" programs in all other programming languages too!
> did you feel that whitespace for lexical scoping made the job of writing the lexer significantly more complex?
Significant indentation requires a more complex lexer because it means the lexical grammar is no longer regular. The lexer can't just be a finite state machine, instead it has to maintain a stack of previous indentation levels.
But I don't think many languages have a regular lexical grammar anyway. Without significant indentation, other features still require the lexer to maintain a stack - e.g. string interpolation (Python's f-strings).
Or you just forbid mixing spaces and tabs in the same indentation sequence, the way most whitespace-sensitive languages seem to end up doing. Or you make a slightly more reasonable rule: spaces may follow tabs, but no tabs may follow a space. That's at least unambiguous.
Oh, that's really elegant! I've got a whitespace sensitive language of my own, and I think I'll change it to use that rule! Thanks!
(Until now, I went with the standard approach: Remember the leading whitespace of the previous line. Then compare with the new line's leading whitespace: If they are the same, then no change in indentation. If the old one is a prefix of the new one, it's an indent. If the new one is a prefix of the old one, it's a dedent. If neither, it's an error)
But it also feels arbitrary and annoyingly restrictive. On top of that there are at least 25 whitespace codepoints in UTF. Should your language really be opinionated about when, where, and in what order (for example) the "mongolian vowel separator" appears?
I mean, obviously that one should only appear within Mongolian text and not within indentation.
To state explicitly what should be implicitly obvious, there is no valid reason (that I'm aware of, I welcome any non-facetious correction) to use any character except U+0009 and U+0020 within indentation. Horizontal Record Separator? Zero-width joiner? Language-specific whitespace characters like your example? All make sense within human text (well, maybe not HRS), but in programming, they should be eschewed in favor of the characters that can be typed in every single keyboard layout in the world. Even languages that don't put spaces between words, such as Thai, still put spaces between sentences (or comma phrases) and therefore keep the space bar in their keyboard layout.
And since mixing tabs and spaces (even between lines, where some lines are tab-indented and some are space-indented) creates problems for whitespace-sensitive language, there's a reason why every whitespace-sensitive language I'm aware of has tended to either outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations, which are usually not available to the validation code running in CI or on other people's machines).
> To state explicitly what should be implicitly obvious, there is no valid reason ...
There doesn't need to be an articulable reason. Or rather there's generally no expectation that a central authority will be able to reliably enumerate such. Everything should default to being permitted and only ever be restricted for good reason.
But since you asked. U+2003 for example carries formatting information. Maybe an editor could be written (or even already exists) that would find that useful. Who is any third party to dictate that?
U+00A0 similarly communicates information about the desired formatting and I can see no reason it would be unreasonable for someone to use it nor why its use should pose a technical challenge to a compiler.
> creates problems for whitespace-sensitive langauge
Does it? That seems like an invented problem to me. You have a running prefix composed of arbitrary whitespace characters. Any change in that prefix is a change in the level of indentation. You can add or remove arbitrary amounts from the end of the prefix. In the event you remove from it the result must exactly match the previous stack level. What's so complicated about this?
> outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations ...
Did you mix up your code points there? It's space that's ambiguous, not tab.
Regardless I think that a compiler worrying about the specifics of text editors or other tooling would be backward information flow and a massive abstraction violation. Semantic meaning is entirely dictated by the compiler, not the other way around. There's no convincing reason (IMO) to impose restrictions that aren't technically necessary or to otherwise needlessly employ solutions that would reduce generalization.
> Did you mix up your code points there? It's space that's ambiguous, not tab.
Space is always the same width, but tab means a variable number of spaces (usually either 4 or 8, but I've seen 3 before) depending on people's editor configuration.
What makes you say that the space character, U+0020, is ambiguous?
How so? In what scenario would you ever need to use a sequence like <tab><space><tab> in indentation in your source code? Let alone using esoteric Unicode whitespace characters for indentation. I think it is perfectly reasonable for the language to make the restriction that indentation must be either all tabs, tabs followed by spaces, or all spaces.
> In what scenario would you ever need to use a sequence like <tab><space><tab> in indentation in your source code?
Writing a lisp in an editor that doesn't do boneheaded things with tabs? That's the only one I've personally run into but lack of imagination is hardly a good excuse to implement arbitrary restrictions.
> Let alone using esoteric Unicode whitespace characters for indentation.
How do you know what's esoteric in other countries? I certainly don't. I'm not an expert in linguistics but I'm sure that people everywhere in the world write computer programs at this point.
> I think it is perfectly reasonable ...
Without any concrete justification? Why would entirely artificial restrictions ever be seen as reasonable?
Right, pointers to strings but yeah. Essentially the whitespace count specifies the stack depth at which a line is to be executed. A decrease in stack depth means all superior levels are terminated.
At a former workplace where most stuff was done in PHP, some colleagues used whitespace very liberally. Like, indentation was just a random amount of whitespace, every line slightly different. Sometimes 2 or more spaces between keywords, etc.
After that experience Python code is like eye-bleach to me.
I meant users of languages ( application programmers) as opposed to compiler programmers, not python programmers specifically, so I'm including devs that use other languages and see in python a tool that they would consume.
This "Python" just plain assumes for keywords: Any "f" is a "for [x] in range[y]" (exactly that, no other for's). Any "w" is a "while". Any "i" is an "if". Any "d" is a "def". Any "p" is a "print("
Nasty, nasty.
(Also nasty is that the code snippets in the article has more comments than the github copy of the "readable" version. You need the article to understand what's going on.)
This is a just a bit too simple for a "Tiny Python". If somebody is willing to allow a few more K's of bytes, I'd love to see at least lists & dicts here--Lisp can do them!
But yes, amazing project! I like that it's human-made :)
https://www.a1k0n.net/2006/09/15/obfuscated-c-donut.html
https://justine.lol/sectorlisp/
https://github.com/xorvoid/sectorc
Edit: I wonder if sectorC could compile python1024
This is not Python, or even within three orders of magnitude of Python.
One could imagine an even smaller subset interpreter. It's an interpreter for a subset of Python, consisting only of the programs that print "Hello World". Since it doesn't do any error checking, for all other programs the output is undefined. Implementing it is very simple: Just ignore the input file, and print "Hello World". As a bonus, it's an interpreter for the subset of "Hello World" programs in all other programming languages too!
</tongue-in-cheek>
As implementer of an interpreter, did you feel that whitespace for lexical scoping made the job of writing the lexer significantly more complex?
Significant indentation requires a more complex lexer because it means the lexical grammar is no longer regular. The lexer can't just be a finite state machine, instead it has to maintain a stack of previous indentation levels.
But I don't think many languages have a regular lexical grammar anyway. Without significant indentation, other features still require the lexer to maintain a stack - e.g. string interpolation (Python's f-strings).
<space><space><tab><space>
is different than
<space><tab><space><space>
So you also have to track the actual sequence of counts of white space used for each level, rather than just a simple count.
(Until now, I went with the standard approach: Remember the leading whitespace of the previous line. Then compare with the new line's leading whitespace: If they are the same, then no change in indentation. If the old one is a prefix of the new one, it's an indent. If the new one is a prefix of the old one, it's a dedent. If neither, it's an error)
To state explicitly what should be implicitly obvious, there is no valid reason (that I'm aware of, I welcome any non-facetious correction) to use any character except U+0009 and U+0020 within indentation. Horizontal Record Separator? Zero-width joiner? Language-specific whitespace characters like your example? All make sense within human text (well, maybe not HRS), but in programming, they should be eschewed in favor of the characters that can be typed in every single keyboard layout in the world. Even languages that don't put spaces between words, such as Thai, still put spaces between sentences (or comma phrases) and therefore keep the space bar in their keyboard layout.
And since mixing tabs and spaces (even between lines, where some lines are tab-indented and some are space-indented) creates problems for whitespace-sensitive language, there's a reason why every whitespace-sensitive language I'm aware of has tended to either outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations, which are usually not available to the validation code running in CI or on other people's machines).
There doesn't need to be an articulable reason. Or rather there's generally no expectation that a central authority will be able to reliably enumerate such. Everything should default to being permitted and only ever be restricted for good reason.
But since you asked. U+2003 for example carries formatting information. Maybe an editor could be written (or even already exists) that would find that useful. Who is any third party to dictate that?
U+00A0 similarly communicates information about the desired formatting and I can see no reason it would be unreasonable for someone to use it nor why its use should pose a technical challenge to a compiler.
> creates problems for whitespace-sensitive langauge
Does it? That seems like an invented problem to me. You have a running prefix composed of arbitrary whitespace characters. Any change in that prefix is a change in the level of indentation. You can add or remove arbitrary amounts from the end of the prefix. In the event you remove from it the result must exactly match the previous stack level. What's so complicated about this?
> outright forbid, or at least discourage, U+0009 and its ambiguous meaning (since its meaning isn't clear until you know people's editor configurations ...
Did you mix up your code points there? It's space that's ambiguous, not tab.
Regardless I think that a compiler worrying about the specifics of text editors or other tooling would be backward information flow and a massive abstraction violation. Semantic meaning is entirely dictated by the compiler, not the other way around. There's no convincing reason (IMO) to impose restrictions that aren't technically necessary or to otherwise needlessly employ solutions that would reduce generalization.
Space is always the same width, but tab means a variable number of spaces (usually either 4 or 8, but I've seen 3 before) depending on people's editor configuration.
What makes you say that the space character, U+0020, is ambiguous?
How so? In what scenario would you ever need to use a sequence like <tab><space><tab> in indentation in your source code? Let alone using esoteric Unicode whitespace characters for indentation. I think it is perfectly reasonable for the language to make the restriction that indentation must be either all tabs, tabs followed by spaces, or all spaces.
Writing a lisp in an editor that doesn't do boneheaded things with tabs? That's the only one I've personally run into but lack of imagination is hardly a good excuse to implement arbitrary restrictions.
> Let alone using esoteric Unicode whitespace characters for indentation.
How do you know what's esoteric in other countries? I certainly don't. I'm not an expert in linguistics but I'm sure that people everywhere in the world write computer programs at this point.
> I think it is perfectly reasonable ...
Without any concrete justification? Why would entirely artificial restrictions ever be seen as reasonable?
in my view, both are the same, both `is` (or ===) an IndentationError raise
Doesn't affect function call stacks though.
Uhhh, no. Sure, it's posited by people who feel they are are forced to use it, but it's basically unlearning other syntax.
Here's a study about people with no experience. They do better with python:
https://www.researchgate.net/publication/262256894_An_Empiri...
When the scala language made whitespace optional, it was very divisive, but now it's extremely well accepted.
After that experience Python code is like eye-bleach to me.