I think I agree with the idea. The numbers could be debated. Mine are much, much lower.
Anecdotal evidence: my first attempt at an Early parser was in Lua. The recogniser went smoothly enough, but the parser was an utter failure. I couldn’t do it. At all. I kept getting stupid errors like “you’re feeding a function to an addition”, or “you’re trying to call a number”… And I had no idea where those errors came from. The size of my little tree search script? 50 lines. I had to rewrite it in Ocaml to even have a chance.
Indeed it wasn’t just static typing. But it remains at the top of my least. My subjective evaluation of what helped me, from the most important to least important, would probably look like the following:
Static typing.
Garbage collection (also present in Lua).
Sum types (especially combined with GC).
Pattern matching.
Parametric polymorphism (a.k.a. generics).
Local type inference (convenient, but by no means mandatory).
Global type inference (it did help, but I’d say marginally so)
I could have done it with just items 1 to 3. The rest just let me get there faster. But yeah, right too for the job: if you’re doing tree traversal OCaml is an obvious candidate.
Couldn’t disagree more! At <100 LOC, I’m almost always begging for types because they can add much needed context for future developers having to touch my scripts. Understanding what values are present in my github.event object currently requires googling GitHub’s REST documentation, where as if the types were laid out for these objects it would be a lot easier to develop with. Actually typing this out has inspired me to look up typings for octokit, so I may port that 88 LOC JS to some TS.
At 1 000 000 LOC, static types are the only sane way
Last year, I wrote a 125 line JavaScript program to allow users to manually select between light or dark color
schemes on the website. The first pass was in plain JavaScript, and it was a struggle.
When I introduced types with TypeScript, it revealed the distinction between
the following concepts:
// ColorScheme represents a concrete color scheme value that can be rendered.
type ColorScheme = "light" | "dark"
// ColorSchemePreference represents a user's choice of color scheme.
type ColorSchemePreference = ColorScheme | "no-preference"
Note that "no-preference" cannot be rendered. But it is a valid color scheme choice, for example, if the user hasn’t chosen a value yet, in which case the program should use a default color scheme value, such as "light".
If I continued without types, the program would likely
muddle through its logic, sans this key distinction, and would have
depended on crude assertions or comments, such as:
// At this point the color scheme c is guaranteed to be a concrete (i.e.
// renderable) value, because we default to "light" when the user has no
// preference.
assert(c === "light" || c === "dark")
renderColorScheme(c)
or poorly used a null value to represent lack of preference:
if (c === null) {
c = "light" // default to "light" when there is no preference
}
renderColorScheme(c)
This is the opinion I’ve come around to, as someone who reaches for Python first for scripts but who has also worked on a large production Python application. Python is great for smaller programs written by individuals. When the codebase gets larger, or when the developer becomes a team of developers, is when it starts being impractical for the types not to be (1) checked automatically by the computer and (2) spelled out in the source code for anyone reading.
After implementing strict mypy compliance in a few repos, I’d probably use something like 5 LOC, 10 LOC and 100 LOC as the limits. By the time you reach 10,000 LOC the chore of introducing types (especially in highly abstract code, like an ORM or dispatcher) becomes huge. I’ve got an ~1,800 LOC Python program (old, from back in the 2.5 days or so) which is much more of a chore to work on than it could be.
I think it’s also a question of when you will experience breakage. Will your code fail during compile time because you missed something that strong typing catches, or do you experience that breakage during runtime? Because the first thing will hopefully find all these spots simply because your compiler will “test” it for you during compilation, and the latter one requires you to actively run (coverage) all possible paths where something could be broken by your type changes.
Which to me essentially feels like nullpointer (un)safety and view-lifecycle state management in android: You change one thing, you have static types, but whether you actually didn’t get the lifecycle wrong, will only show up if you actually navigate to that part of your application (and push some buttons or rotate your android app). Same goes for possible nullpointer exceptions.
Of course you could test this out via Unit-Tests and coverage, but you will always miss a spot, and static typing will at least find this stuff without writing tests for function signature expectations. (Let’s not talk about runtime code loading, type erasure and other things that essentially break static types.)
Honestly I find that even the 100 LOC static types are helpful, even if only to provide better editor support for the APIs I’m using and to make up for poor documentation (e.g., even the Python standard library documentation lists things like “file-like object” with no specifics about the APIs that are actually consumed). Plus even at the 100 LOC level, I’m always forgetting little things like awaiting async functions.
Also it’s worth noting that Python’s static types are more grating than necessary (both from a syntax perspective but also loading type stubs, etc). TypeScript is a lot more pleasant to work in (I say this as a Python dev of 15 years and very occasional JS/TS dev).
But does enforcing strict rules work in practice? … I want a system where the chances of making mistakes are minimal. I can achieve that with unit tests and CI/CD pipelines. But type safety provides me that by default.
I often say that type checks are the real bottom level of the test pyramid. They’re finer grained and more numerous than unit tests.
Type checks reduce the need for unit tests. This effect is greatest when types make invalid cases inexpressible. But that still means there is a point at which the team must be diligent.
I remember a JS project that I worked on that prided itself on 100% unit test coverage, and I couldn’t shake the feeling coming from heavily typed languages, and reading some of these unit tests that were meant to produce 100% coverage, that they reduced themselves to rote manual implementation of basically type checking. What an enormous amount of work to “avoid” type checking.
The main advantage I see to those rote unit tests is it lets you confirm that your function’s API is… usable? You have to prove by construction that the arguments you pass in are buildable, for example.
This seems like a weird point, but I have witnessed people kind of build up a type hierarchy through typing out stuff and then at the end realize that they have some weird circular reference in their types or something that makes their functions no good.
I mean I still put type hints, if only because it’s like comments! Even in the most dynamic language in the world, you’re gonna say what you pass in!
I think one of the most important properties of type checks, in statically-typed languages, is that you can’t opt out of them. I have to imagine there are very few engineering organizations that enforce 100% test coverage 100% of the time. If someone says “we don’t need static types because we check everything with tests,” and then they don’t, in fact, check everything with tests—well, where does that leave you?
I don’t really agree, in statically-typed languages there is a variety of design choices you can make from “barely typed” to “typed in a very fine-grained way”. For example you could represent all your data as String with weird accessor functions, and there you go.
Having a static type system lets you express more static policies than “none”, but it doesn’t in itself ensure that your type system enforces all the correctness properties that it could. And this is right, because sometimes there are properties that can be expressed, but at a large cost in convenience and usability, so we explicitly opt out.
(For example, most modern statically-typed languages have horrible ways to encode some partial reasoning about units of measure, and in most of them these horrible encoding are never used because they are unusable in practice.)
I agree with you that there will always be program behaviors that it’s impractical or impossible to enforce at the type level. (I have a moderate amount of experience with Haskell, which goes farther than most languages in this respect, albeit that it doesn’t have the dependent-typing features of some languages.)
The question I’m interested in is, “how many guarantees can we make about a program before we run it?” Adding a lot of tests pushes up the ceiling of the answer. In other words, writing tests allows you to make more and more guarantees that the program will behave correctly. I think of static types as raising the floor. Types do not by themselves give you as many guarantees as you could potentially get by writing tests. But types enforce a more robust lower bound on, “what’s the stupidest thing that could surprise me at runtime?”
As a concrete example, suppose you have a function that takes some text and encodes it (& to & and so on) so that it can be safely displayed on a web page. You could quite reasonably decide to use String for both the input and output types instead of going through the trouble of defining a separate HTML type. In this case, Haskell wouldn’t stop you from accidentally calling the function twice and turning & into &amp;. But it would stop you from calling that function with a null. If you were doing the same thing in Python and you skipped writing tests for that function—or even if you just forgot one test case!—you could pass None to the function and boom, crash at runtime. That’s a sillier problem, in my opinion.
In other words, writing tests allows you to make more and more guarantees that the program will behave correctly. I think of static types as raising the floor. Types do not by themselves give you as many guarantees as you could potentially get by writing tests. But types enforce a more robust lower bound on, “what’s the stupidest thing that could surprise me at runtime?”
most modern statically-typed languages have horrible ways to encode some partial reasoning about units of measure, and in most of them these horrible encoding are never used because they are unusable in practice.
100%, and not just units of measure. Going beyond, this is a string, or this is an 8 bit number in most static typing systems is either not possible or an exercise in headache-inducing wackiness that nobody ever bothers with. Sort of makes you think, what’s the point again?
(For example, most modern statically-typed languages have horrible ways to encode some partial reasoning about units of measure, and in most of them these horrible encoding are never used because they are unusable in practice.)
As an OCaml person, what do you think of F#’s take on units of measure?
I don’t have experience using it, but I think that it is nice work. (Certainly much better than the encodings I mentioned.) F# has nice features geared towards scientific domains or data analysis (type providers also come to mind, or the nice libraries for probabilistic programming, etc.). It is partly a matter of where the language spends its complexity / feature-weight budget.
I quite like this analogy because it nicely frames my gripe with types in many cases: just like unit tests, 100% coverage is often overkill. I like types and I like tests but I most prefer to choose when to use each. Type inference often feels quite nice relative to languages where you had to specify all of the types, sometimes twice. However type inference can also be quite annoying when it provides 100% type coverage in places it really isn’t needed. This is exacerbated by overly specific types in many cases. e.g. the rust experience would be much nicer if the default was to infer all numbers as some sort of BigInteger or Decimal type that escaped the perils of overflow and NaN != NaN. When you need those constraints you know it but when you don’t need them they become a burden for the unfamiliar and an annoyance for the familiar.
The main problem with big codebases in Python is not the lack of types, it’s the lack of discipline (of well tested codebase, good documentation, high quality code). Types enforce that discipline, so you end up with a more understandable codebase by default, but spaghetti code is spaghetti code, no matter what language and you can write high quality code in Python too.
Python literally adds type checking in order to allow codebases to scale. This is like being mad bash because your shell script is too long: at a certain point you need to change your practices. You could rewrite in a completely different language, or you could incrementally add type checking if you’re using Python (or any other optionally typed language).
This is also literally a consequence of having poor test coverage, which you can’t blame Python for either.
Mature, large codebases need different things from quick scripts, and potentially need quite different things from things that you don’t know will stick around (like a startup product). As some guy observed, there is no silver bullet for productivity.
If you’re going to argue “the type checker might have a bug”, then you’re kind of undermining your own point – if that’s an acceptable argument in one language it’s an acceptable argument in any language, and everything you think you need to do to work around a dynamically-typed language now also becomes mandatory in every language.
You always need to write unit tests. But if the argument is that a non-Python language needs to be used in order to gain the alleged benefits of static typing, while Python’s own static typing tools can’t be used in case they might have bugs, this raises questions about why anything else would be trusted. I could just as easily say I refuse to use C# because the .NET compiler might contain type-checker bugs.
That’s not his argument. His argument is that mypy will accept type-unsafe python code. From his linked tweet:
I recently discovered pytype, a static type analyser for Python from Google that figures out types from the static analysis. In contrast, mypy relies on annotations.
Following code passes mypy, but pytype recognises that str.join requires an iterable, not individual strings!
I still don’t think it gets out of the underlying issue, which is that it still ends up being a subjective, not an objective, argument and choice.
There’s a whole spectrum of stuff that might or might not be caught depending on the language and in some cases the toolchain used to check it. And from there people can easily stake out their subjective preferences for which tradeoffs they’ll accept and which ones they won’t. And that can be interesting and fruitful as long as people are being honest about it.
But this post was just another recycling of the same tired old “dynamic bad, static good” tropes – you can’t refactor dynamic code! all large projects require static typing! you never can know anything! – that have already been done a million and one times before. The million and second didn’t add anything useful, that I can tell.
One of these days I may just write the inverse post (large successful things like Facebook/Instagram and YouTube in dynamic languages indicate that static languages don’t scale with size of team or codebase; static languages lack of proper documentation beyond argument signatures, and so existing static codebases are fundamentally impossible to get to grips with; refactoring was invented in a dynamic language, can static language IDEs even support it at all? etc.) and see how much I can get away with before people realize the gimmick.
This is kind of a silly argument, because it assumes that the dynamism of the language precludes type-safety. That is simply not the case, it means that type-checks happen at runtime rather than during compilation.
Moving type-checks to compilation makes them more robust, but at the cost of the power you gain from dynamic language features. It’s a tradeoff, not a straightforward benefit. There are things you simply cannot do in a statically-typed language.
The first issue I faced was understanding the code. It took a sweet amount of time to figure out the kind of objects, things certain functions were receiving and what they were doing with them. The code did have some unit tests, but the coverage was poor. So I had to guess, make changes and test the code at many places.
The real problem here is that you allowed poor coding practices to ship. If you care enough about correctness to switch languages, then you care enough to add systems that enforce this. It’s a false dichotomy.
Years of coding in Go gave me this comfortable feeling: if it compiles, it works.
No the fuck it doesn’t are you kidding me. You are fully able to write incorrect Go code. You still need to write tests in Go.
I would even argue that “if it compiles, it works” is more realistic in Python + mypy than it is in Go:
You can’t accidentally use a nil pointer in typed Python.
You can’t forget to initialize a variable in typed Python. (Yes in Go the variable is initialized with a zero value, but there is a legitimate amount of cases where the zero-value is not what you want as a default).
Similarly, if you add a field to a class in Python, all code that initializes that class is going to fail type-checking until you initialize the field there.
This is of course assuming that everyone is very disciplined about using types and not abusing Any, which may be a strong assumption.
There is one important distinction that I see here, and that burned me several times. If mypy fails, I can ignore it and still
run by that python code, only to possibly fail later. If it fails in go, there won’t be anything for me to run.
Nevertheless, I agree with you that there are some classes of errors that could be caught and prevented with mypy.
Kinda funny, it’s the opposite for me: when trying things out the flexibility of bypassing type checks helps, and CI ensures type checking is never ignored in prod.
In find the Go compiler especially annoying when experimenting because it won’t even let me run code with unused variables and imports, they really don’t matter when experimenting.
The core argument for compile time type safety is that sooner or later you always allow poor coding practices to ship. The larger codebase and the longer lived it is the more likely that there are dangers lurking in there for the next refactor that you won’t discover until you hit production. Compile time checks can’t eliminate them totally but they can reduce their frequency considerably. For some people and projects that’s a really valuable attribute.
The argument being made here is that Go’s types are preferable to Python because developers can’t be trusted to test their code. They are literally suggesting that types are a replacement for test coverage.
This is delusional. Types reduce the amount of testing needed, but they will never replace it. If your developers are this unprofessional, I guarantee they will find a way to fuck up in Go as well.
The argument being made here is that Go’s types are preferable to Python because developers can’t be trusted to test their code. They are literally suggesting that types are a replacement for test coverage.
No, I am not. Could you tell me how did you get that impression? I’d be happy to add a correction
Unit testing is the way to go. However, this is not always possible, for instance, when enforcing rules within a team is difficult or when inheriting a legacy project. But does enforcing strict rules work in practice? You could set up CI/CD pipeline enforcing 100% code coverage, but that will affect team productivity and surely piss off people.
So your argument is that Python is a bad choice because
developers don’t unit test
you can’t enforce CI/CD
attempting to enforce correctness will piss people off
This is a people problem, not a technical one. Using a static type system will not solve this lack of professionalism.
I am attempting to give them the benefit of the doubt; if they intended to say that type-safety is equivalent to 100% code coverage, then that would be even more nonsensical.
You’re pretty obviously not giving him the benefit of the doubt. The “benefit of the doubt” reading is “getting and enforcing 100% code coverage is harder than using static types in conjunction with less-than-100% code coverage.”
Nobody anywhere is suggesting that 100% code coverage is reasonable. This is a dumb strawman about testing. I’m pretty sure that you know this.
When somebody writes a blog post that tells me the things I have spent the better part of a decade-plus career doing is impossible, I am gonna call bullshit.
I have literally gotten in arguments with people who definitely think that 100% code coverage is both reasonable and necessary. Perhaps it’s not actually a strawman but something they have encountered in their own career?
that the dynamism of the language precludes type-safety. That is simply not the case, it means that type-checks happen at runtime rather than during compilation.
Type checks that happen at runtime are not useful. In fact, they shouldn’t even be called types at that point. Types are for determining if static code terms are valid or not. And to even say this, it sounds like you didn’t read the article:
The first issue I faced was understanding the code. It took a sweet amount of time to figure out the kind of objects, things certain functions were receiving and what they were doing with them.
Dynamic types don’t help with that. And that’s not a “silly” argument. Do you realize that’s insulting?
That is simply not the case, it means that type-checks happen at runtime rather than during compilation.
The question is when a check gives you the most value. You can say an exception is a unit test that runs in production, but I prefer to run my tests as early as possible, with reasonable effort. For the class of invariances that can conveniently be expressed and checked using a type system, that’s what I prefer. For those that can be caught by unit tests, that’s the best. For those that require integration tests… etc. All the way to load tests using realistic fake traffic, if that’s needed.
if it compiles, it works.
All depends on what you mean by “works”. Of course, in any moderately complex system, there are an infinite number of intended behaviors that can’t be expressed through a type system (or unit tests). The closest thing I’m aware of is Elm, where the combination of a good (and rather simple) type system and very limited runtime capabilities make it so that if it compiles, it’s extremely rare that there are runtime exceptions. It brings me a lot of joy to be able to skip out on the part of the development cycle where I finish editing some code and wait to see if the program crashes or not. It’s great to just skip ahead to seeing if the higher-order behaviors are as I expected or not. It really does save me time and, most importantly, helps me keep mental focus and enthusiasm.
For the class of invariances that can conveniently be expressed and checked using a type system, that’s what I prefer. For those that can be caught by unit tests, that’s the best.
This is perfectly reasonable, but I will point out that I follow this approach in dynamic languages as well. Dynamic languages are capable of exactly the same constraints (in fact, they can be even more sophisticated), but you have to go about it differently.
Type-checking at compile time comes at a cost, the robustness of the checks means you are giving up flexibility to do certain things. This may be a worthwhile tradeoff for your situation — but it should be made intentionally.
I want to push back against this notion that you need static types to achieve this level of correctness. You do not. Don’t blame your tools for unprofessionalism.
the robustness of the checks means you are giving up flexibility to do certain things
Could you show me some affect of a program which is possible to produce with dynamically type checked tooling but not statically typed tooling?
Don’t blame your tools for unprofessionalism.
I’d argue that the industrial benefits of static typing are so obvious and accessible at this point in time that it’s unprofessional to operate under the assumption that “comprehensive” testing can replace the many roles of static tooling.
Many other examples of statically typed languages with REPL’s exist but this one is my favorite as it’s almost designed around progressive interactions with the REPL.
Moving type-checks to compilation makes them more robust, but at the cost of the power you gain from dynamic language features …
But the Python type hints are extremely flexible and gives you lots of power to take advantage of all the great dynamic language features of Python.
They are also not an all-in-or-nothing kind of thing. If you have code that needs to be super dynamic and creative and type hints get in the way then you can just as easily decide to not use them there.
(Or - in many cases you can usually create an “outer” layer of typed API while the inside is all the Python magic you want)
I believe both our arguments are perfectly compatible here. I don’t really consider Python’s type-hints to be in the same category as Go’s type checking, I am attempting to speak more broadly about dynamic vs static type systems and not get bogged down in a specific implementation.
The code did have some unit tests, but the coverage was poor.
Well, there’s your first problem. Yes, if you compare code that you wrote in a dynamically typed language, when you were inexperienced and in a hurry, with code that you wrote in a statically typed language, when you were more experienced and had more people collaborating with you, guess which one will be better. It’s not an indictment of dynamic languages though, but rather of your methodology. I know this is not a scientific article but you gonna make arguments, they should be good.
A language with type makes a lot of difference when refactoring; one glance at the signature, and I know what to do.
Dynamic languages have pros and cons like anything else. I might even agree that they might have more cons than certain statically typed languages. But there’s no technology that can prevent writing bad code.
Without good processes and experience, your code is gonna suck in any language. It might be marginally better with types, maybe, but even that is not a given.
Unless your hypothetical language has immutable parameters, the only thing I can assume there is that fb might be modified by frobnigate() (less so j and s, because most languages these days are pass-by-value; otherwise if the language is pass-by-reference, then I’d be worried about all the parameters). But aside from that, I’d have to look at the code (most likely documentation is sparse).
I can tell you just from the type signature that any element in the result list must have come from applying the function argument to one of the elements of the input list.
The idea of shipping a massive system to users without strict types seems insane to me. How do you know your tests cover every viable path through your code? How on earth do you maintain a legacy codebase with 0 types and 0 documentation? How do you ensure any code path is valid if values can be and type at any time? I wish these were strawmen, but I keep running into abandoned (and fresh) Python apps that have these problems.
With typed languages, I used to think, ugh, I’d have to define types everywhere, make struct (or classes) in advance, and sometimes make wild guesses because those things were unknown or not finalised. All of these are valid complaints. If I am prototyping in Rust, I will take a lot of time. With Python, I can ship fast. But only once.
I hate how discourse for static vs dynamic typing pits the two extremes against each other. TypeScript and other gradual type systems are a great middle ground: gradually add types where your team sees fit. You could completely ignore types while developing and make adding type signatures a blocker for merging.
The idea of shipping a massive system to users without strict types seems insane to me. How do you know your tests cover every viable path through your code?
This is one reason I ran away screaming from webdev: everyone wanted to do exactly this, complained that test times were insane (because they didn’t want to take the time to design for testability), and complained that upgrading their framework was painful (because they coupled it directly to the framework). Doing otherwise was taboo, something something not delivering business value.
They were so intent on “disruption” they didn’t have time to stop and think about what would enable them to do that as effectively as possible.
because they didn’t want to take the time to design for testability
Keep in mind that this means very different things in dynamically-typed languages than it does in statically-typed languages. Statically-typed languages basically force you into Enterprise Code™ patterns because of the inflexibility of the language – you have to do dependency injection and inversion of control and five dozen “decoupled” layers of abstraction between you and the world, because the language itself is incapable of letting you do anything else.
But in dynamically-typed languages, many of those patterns stop being relevant because the language-level constraints that inspired the patterns are not present. Consider, for example, the unittest.mock module in the Python standard library: a gigantic swathe of “testability” patterns from statically-typed languages go out the window because of this one tool, which in turn can only exist because of high runtime dynamism in the language itself.
Meanwhile, I’m not sure why “test times were insane” is a counterargument to dynamic typing, considering how many times I hear that people are desperate for compiler speedups because of the multi-hour build times for their supposedly well-engineered software.
Another middle ground is Crystal which I’m using more and more recently where I’d use Python otherwise. There’s enough type inference that I add a type signature maybe once every 50+ lines. Meanwhile all the rest of the code is automagically typed and the compiler will tell me where I made mistakes.
(Most of the typing necessary is for collections, where… yeah, they come extremely useful and stop accidents)
How do you know your tests cover every viable path through your code?
By writing unit tests, the same way you’d do it in a statically-typed language. I’ve never seen a statically-typed language that automatically derives prose documentation and 100% path coverage from the type declarations, so I’m not sure why static typing is supposed to affect this.
How on earth do you maintain a legacy codebase with 0 types and 0 documentation?
Well, I hate to be the bearer of bad news, but legacy projects in any language tend not to come with useful documentation. And given how easy it is to do things like accept a HashMap of “arguments” in Java and Java-like languages, or to just declare everything as interface in Go, I don’t really see how the type signatures are expected to be a panacea. And if you think there aren’t significant real-world real codebases doing those things, you’ve got another think coming – static types don’t and didn’t do anything to prevent that.
How do you ensure any code path is valid if values can be and type at any time?
You test the paths you support. If a client of your code does something wrong, their tests will break. Not because you wrote tests to exhaustively type-check your arguments, but because passing the wrong type of argument tends to throw a runtime – and unit testing is runtime! – error anyway.
I keep running into abandoned (and fresh) Python apps that have these problems.
It’s possible to write terrible undocumented abandonware in any language.
It is interesting that most comments here do not say anything about developer experience. Type hints in Python don’t just prove your code is more correct. They also power excellent tools like PyLance or python-lsp-server. In my opinion there is nothing more productive than hitting “.” and seeing what properties and methods are on a thing you are going to access. I can make decisions in a matter of seconds instead of vague guess work or assumptions or having to grep through code to find whatever it is that I am working with.
Type hints improve the developer experience 10x. At least. Whether you are in a team of 50 or a solo developer. Many major libraries now have type hints - hacking Python was already productive, but it skyrockets with type hints and a good IDE to show you choices/details and even documentations right there.
I’ve had this same experience with Python. FWIW Python’s standard library has been better than JavaScript’s so you’d be less likely to have cannot read property 'foo' of undefined since dictionaries have a get method and None is only value to represent null or undefined. I think this gave Python an advantage in the earlier days of the dynamic language showdown between NodeJS and Python. At least in my experience at work there would be way more bugs in NodeJS projects than in Python ones even without types.
Today, I don’t feel confident writing code in Python, even with types. They feel clunky to me with their syntax. It seems that types are not even that widely adopted in Python libraries. TypeScript on the other hand has become way more popular within the JS community and is a nicer syntax for writing JavaScript code with types. Even though I prefer Python as a language, TypeScript’s type system feels better to me than Python’s. TypeScript also has libraries like fp-ts and zod which make it easy to parse and validate input and keep type errors at the edge of your program.
Unfortunately, to me at least, Python feels like a dead language. I no longer want to use it, but I do miss a lot of things it has (comprehensions, dataclasses, SQLAlchemy & alembic, pytest, etc.).
Years ago, I started thinking about writing a blog post entitled “How Python became my third favorite language.” The idea was to list all the things that had made it my favorite language back in ~2006 and then write about how Go and JavaScript had usurped Python by surpassing it in those areas. I’ve never written the post in part because of laziness and in part because it’s a bit of a downer, but what you’re saying about Node/TypeScript is definitely one element. I don’t even use TypeScript myself, but because other people do, I get automatic type inference to the frameworks I call into. It has been a while since I tried to use static types in Python, but my last attempt bailed pretty quickly because some popular library wasn’t compatible with what I was trying to do, so I just gave up.
I think a huge amount. IMO NodeJS (and then Go) stole a lot of marketshare of new developers away from Python because setting up Python correctly (with virtualenv) is such a nightmare, especially on Macs. It’s a lot more simple to just install the runtime and then install the packages for a project locally instead of some path relative to where the language is installed which is is stored in a shell env var.
During development itself, not much. The packaging issues affect the initial project setup and big migrations. Otherwise, you can mostly ignore it and just add packages like you always do and update them like you always do. Put the commands in the readme if needed.
So I’d go with some impact in general, and almost none for “this productivity” as described in the post.
I think it’s on point that the main problem is with refactoring. Adding new code is not hard, because it requires testing just the freshly added code, which can be done manually. But refactoring can affect many places all over the codebase, and requires finding them all.
I believe this is because the current set of dynamically typed languages we have are at a local maximum, where the type information is perfectly clear and fully introspectable at runtime (even more so than statically-typed, compiled languages), but the language has no tools to introspect or take advantage of these types, because the development context is outside of the runtime where such types are unavailable. And for that matter, I don’t think it’s types that we actually wish for either; what we’re looking for is interfaces that allow us to ask questions to objects and receive responses, and concrete types happen to be one common use of such a construct.
From these two points, I envision the next step in dynamically typed programming to be a system where the development workspace executes in the same context of the program being modified. The workspace allows you to modify and extend a program while it’s running, and the type information that’s available at runtime guides the programmer in creating interface definitions for both documentation and as guides while editing.
Obviously, such a system needs to have proper isolation mechanisms in order to prevent the workspace from being affected by any errors that occur within the program. A process abstraction would be in order here, as well as a way to prevent untrusted programs from doing things that the user didn’t intend (a capability based system sounds nice).
Either Lisp or Smalltalk (because the two paradigms are isometric). I’m working on a version of the Self programming language that I want to eventually get to the point I’ve described.
TL;DR: Typing is a useful tool making software cheap to maintain in the long term, which the author now needs.
Strong typing has a small cost for the individual developer and provides a benefit in helping coding between authors or across long time spans. The author has changed jobs from a goal of cheap in the short term to cheap in the long term and now extols typed languages.
As always when the type wars come up, I want to remind people that the evidence for statically typed languages improving software quality is limited and ambiguous.
The whole ‘i have unit tests to cover me for typing and refactoring’ was always a bad excuse. It was for Ruby ,where I think it started to be a meme, and it is for Python.
It was strongly believed and advocated by many at the time. It’s actually shocking to see such a large majority now on the side of strong typing. (I was always on the strong typing side, so I’m not shocked that strong typing is so powerful … I’m shocked at how large a percentage of developers have come to that conclusion).
Apparently, this won’t be like the spaces/tabs and emacs/vi(m) religious wars … we might reach mainstream consensus on this one.
I’ve expressed a similar sentiment before. I’ve done most of my work in C, C++, Java and Python. And for much of my career, I’ve felt most productive in C++, due in no small part to compile-time type checking.
Over the past few years, I’ve changed my opinion a little bit. Python’s type system lets me test ideas and figure things out very quickly. And once I have, its type hints give me almost all of the compile time checking I have always leaned on in C++ (and later Java).
The ability to not need to write down the types to try out an idea, but to then be able to write down the types and have them warned/enforced before I ship has proven quite productive and powerful for me. And when I stack on the ability to have pydantic use them too, it feels like a super power. So as a very C++-biased person, I have lots good to say about python’s typing. Which means I disagree with this post in the large, I suppose, even as it does resonate with me.
I kind of feel the same way even though I’ve written a very small amount of Go. While I do try to add type hints wherever possible to aid my future self, it’s nowhere near as close to the level of helpfulness as, for example, in Go where VS Code just says flat out that doing X in place Y would definitely be incorrect for Z to still be true. This is can be alleviated somewhat with Pylance’s strict mode, if I’m not mistaken, which will emulate the same experience as in Go, but if you haven’t gone the strict type hinting route from the very start then it’s just going to be a nuisance with you trying to retroactively type hint properly all the old things in order for the new additions to be valid.
At 100 LOC, static types are mostly in the way
At 10 000 LOC, static types help you along the way
At 1 000 000 LOC, static types are the only sane way
I think I agree with the idea. The numbers could be debated. Mine are much, much lower.
Anecdotal evidence: my first attempt at an Early parser was in Lua. The recogniser went smoothly enough, but the parser was an utter failure. I couldn’t do it. At all. I kept getting stupid errors like “you’re feeding a function to an addition”, or “you’re trying to call a number”… And I had no idea where those errors came from. The size of my little tree search script? 50 lines. I had to rewrite it in Ocaml to even have a chance.
As I said, my own numbers are much lower.
I believe that Ocaml shines for writting parser/compiler (and especially parser/compiler of Ocaml). maybe we can say the right tool for the right job
Indeed it wasn’t just static typing. But it remains at the top of my least. My subjective evaluation of what helped me, from the most important to least important, would probably look like the following:
I could have done it with just items 1 to 3. The rest just let me get there faster. But yeah, right too for the job: if you’re doing tree traversal OCaml is an obvious candidate.
Yeah, sum types and exhaustive pattern matching are a real boon for writing parsers. The latter implies type checking.
Couldn’t disagree more! At <100 LOC, I’m almost always begging for types because they can add much needed context for future developers having to touch my scripts. Understanding what values are present in my
github.eventobject currently requires googling GitHub’s REST documentation, where as if the types were laid out for these objects it would be a lot easier to develop with. Actually typing this out has inspired me to look up typings for octokit, so I may port that 88 LOC JS to some TS.imo:
Last year, I wrote a 125 line JavaScript program to allow users to manually select between light or dark color schemes on the website. The first pass was in plain JavaScript, and it was a struggle. When I introduced types with TypeScript, it revealed the distinction between the following concepts:
Note that
"no-preference"cannot be rendered. But it is a valid color scheme choice, for example, if the user hasn’t chosen a value yet, in which case the program should use a default color scheme value, such as"light".If I continued without types, the program would likely muddle through its logic, sans this key distinction, and would have depended on crude assertions or comments, such as:
or poorly used a null value to represent lack of preference:
This is the opinion I’ve come around to, as someone who reaches for Python first for scripts but who has also worked on a large production Python application. Python is great for smaller programs written by individuals. When the codebase gets larger, or when the developer becomes a team of developers, is when it starts being impractical for the types not to be (1) checked automatically by the computer and (2) spelled out in the source code for anyone reading.
After implementing strict mypy compliance in a few repos, I’d probably use something like 5 LOC, 10 LOC and 100 LOC as the limits. By the time you reach 10,000 LOC the chore of introducing types (especially in highly abstract code, like an ORM or dispatcher) becomes huge. I’ve got an ~1,800 LOC Python program (old, from back in the 2.5 days or so) which is much more of a chore to work on than it could be.
You said it better in just three lines. I am gonna steal your comment!
I think it’s also a question of when you will experience breakage. Will your code fail during compile time because you missed something that strong typing catches, or do you experience that breakage during runtime? Because the first thing will hopefully find all these spots simply because your compiler will “test” it for you during compilation, and the latter one requires you to actively run (coverage) all possible paths where something could be broken by your type changes.
Which to me essentially feels like nullpointer (un)safety and view-lifecycle state management in android: You change one thing, you have static types, but whether you actually didn’t get the lifecycle wrong, will only show up if you actually navigate to that part of your application (and push some buttons or rotate your android app). Same goes for possible nullpointer exceptions.
Of course you could test this out via Unit-Tests and coverage, but you will always miss a spot, and static typing will at least find this stuff without writing tests for function signature expectations. (Let’s not talk about runtime code loading, type erasure and other things that essentially break static types.)
I may need to steal your comment. Thanks
Honestly I find that even the 100 LOC static types are helpful, even if only to provide better editor support for the APIs I’m using and to make up for poor documentation (e.g., even the Python standard library documentation lists things like “file-like object” with no specifics about the APIs that are actually consumed). Plus even at the 100 LOC level, I’m always forgetting little things like
awaiting async functions.Also it’s worth noting that Python’s static types are more grating than necessary (both from a syntax perspective but also loading type stubs, etc). TypeScript is a lot more pleasant to work in (I say this as a Python dev of 15 years and very occasional JS/TS dev).
I often say that type checks are the real bottom level of the test pyramid. They’re finer grained and more numerous than unit tests.
Type checks reduce the need for unit tests. This effect is greatest when types make invalid cases inexpressible. But that still means there is a point at which the team must be diligent.
I remember a JS project that I worked on that prided itself on 100% unit test coverage, and I couldn’t shake the feeling coming from heavily typed languages, and reading some of these unit tests that were meant to produce 100% coverage, that they reduced themselves to rote manual implementation of basically type checking. What an enormous amount of work to “avoid” type checking.
The main advantage I see to those rote unit tests is it lets you confirm that your function’s API is… usable? You have to prove by construction that the arguments you pass in are buildable, for example.
This seems like a weird point, but I have witnessed people kind of build up a type hierarchy through typing out stuff and then at the end realize that they have some weird circular reference in their types or something that makes their functions no good.
I mean I still put type hints, if only because it’s like comments! Even in the most dynamic language in the world, you’re gonna say what you pass in!
I think one of the most important properties of type checks, in statically-typed languages, is that you can’t opt out of them. I have to imagine there are very few engineering organizations that enforce 100% test coverage 100% of the time. If someone says “we don’t need static types because we check everything with tests,” and then they don’t, in fact, check everything with tests—well, where does that leave you?
I don’t really agree, in statically-typed languages there is a variety of design choices you can make from “barely typed” to “typed in a very fine-grained way”. For example you could represent all your data as
Stringwith weird accessor functions, and there you go.Having a static type system lets you express more static policies than “none”, but it doesn’t in itself ensure that your type system enforces all the correctness properties that it could. And this is right, because sometimes there are properties that can be expressed, but at a large cost in convenience and usability, so we explicitly opt out.
(For example, most modern statically-typed languages have horrible ways to encode some partial reasoning about units of measure, and in most of them these horrible encoding are never used because they are unusable in practice.)
I agree with you that there will always be program behaviors that it’s impractical or impossible to enforce at the type level. (I have a moderate amount of experience with Haskell, which goes farther than most languages in this respect, albeit that it doesn’t have the dependent-typing features of some languages.)
The question I’m interested in is, “how many guarantees can we make about a program before we run it?” Adding a lot of tests pushes up the ceiling of the answer. In other words, writing tests allows you to make more and more guarantees that the program will behave correctly. I think of static types as raising the floor. Types do not by themselves give you as many guarantees as you could potentially get by writing tests. But types enforce a more robust lower bound on, “what’s the stupidest thing that could surprise me at runtime?”
As a concrete example, suppose you have a function that takes some text and encodes it (
&to&and so on) so that it can be safely displayed on a web page. You could quite reasonably decide to useStringfor both the input and output types instead of going through the trouble of defining a separateHTMLtype. In this case, Haskell wouldn’t stop you from accidentally calling the function twice and turning&into&amp;. But it would stop you from calling that function with a null. If you were doing the same thing in Python and you skipped writing tests for that function—or even if you just forgot one test case!—you could passNoneto the function and boom, crash at runtime. That’s a sillier problem, in my opinion.This is a brilliant way of putting it.
100%, and not just units of measure. Going beyond, this is a string, or this is an 8 bit number in most static typing systems is either not possible or an exercise in headache-inducing wackiness that nobody ever bothers with. Sort of makes you think, what’s the point again?
As an OCaml person, what do you think of F#’s take on units of measure?
I don’t have experience using it, but I think that it is nice work. (Certainly much better than the encodings I mentioned.) F# has nice features geared towards scientific domains or data analysis (type providers also come to mind, or the nice libraries for probabilistic programming, etc.). It is partly a matter of where the language spends its complexity / feature-weight budget.
I quite like this analogy because it nicely frames my gripe with types in many cases: just like unit tests, 100% coverage is often overkill. I like types and I like tests but I most prefer to choose when to use each. Type inference often feels quite nice relative to languages where you had to specify all of the types, sometimes twice. However type inference can also be quite annoying when it provides 100% type coverage in places it really isn’t needed. This is exacerbated by overly specific types in many cases. e.g. the rust experience would be much nicer if the default was to infer all numbers as some sort of BigInteger or Decimal type that escaped the perils of overflow and NaN != NaN. When you need those constraints you know it but when you don’t need them they become a burden for the unfamiliar and an annoyance for the familiar.
The main problem with big codebases in Python is not the lack of types, it’s the lack of discipline (of well tested codebase, good documentation, high quality code). Types enforce that discipline, so you end up with a more understandable codebase by default, but spaghetti code is spaghetti code, no matter what language and you can write high quality code in Python too.
Python literally adds type checking in order to allow codebases to scale. This is like being mad bash because your shell script is too long: at a certain point you need to change your practices. You could rewrite in a completely different language, or you could incrementally add type checking if you’re using Python (or any other optionally typed language).
This is also literally a consequence of having poor test coverage, which you can’t blame Python for either.
Mature, large codebases need different things from quick scripts, and potentially need quite different things from things that you don’t know will stick around (like a startup product). As some guy observed, there is no silver bullet for productivity.
What about Mypy? Anyway, I share the sentiment that types are better in the long term for programming.
I covered in this point in the post, even with mypy, you can have bugs. Pytype by Google seems to be better. Or may be one should use both?
If you’re going to argue “the type checker might have a bug”, then you’re kind of undermining your own point – if that’s an acceptable argument in one language it’s an acceptable argument in any language, and everything you think you need to do to work around a dynamically-typed language now also becomes mandatory in every language.
Yes, I think the author would still agree that you need to write unit tests even if you have types.
You always need to write unit tests. But if the argument is that a non-Python language needs to be used in order to gain the alleged benefits of static typing, while Python’s own static typing tools can’t be used in case they might have bugs, this raises questions about why anything else would be trusted. I could just as easily say I refuse to use C# because the .NET compiler might contain type-checker bugs.
That’s not his argument. His argument is that mypy will accept type-unsafe python code. From his linked tweet:
I still don’t think it gets out of the underlying issue, which is that it still ends up being a subjective, not an objective, argument and choice.
There’s a whole spectrum of stuff that might or might not be caught depending on the language and in some cases the toolchain used to check it. And from there people can easily stake out their subjective preferences for which tradeoffs they’ll accept and which ones they won’t. And that can be interesting and fruitful as long as people are being honest about it.
But this post was just another recycling of the same tired old “dynamic bad, static good” tropes – you can’t refactor dynamic code! all large projects require static typing! you never can know anything! – that have already been done a million and one times before. The million and second didn’t add anything useful, that I can tell.
One of these days I may just write the inverse post (large successful things like Facebook/Instagram and YouTube in dynamic languages indicate that static languages don’t scale with size of team or codebase; static languages lack of proper documentation beyond argument signatures, and so existing static codebases are fundamentally impossible to get to grips with; refactoring was invented in a dynamic language, can static language IDEs even support it at all? etc.) and see how much I can get away with before people realize the gimmick.
@drmaciver beat you to it :P
This is kind of a silly argument, because it assumes that the dynamism of the language precludes type-safety. That is simply not the case, it means that type-checks happen at runtime rather than during compilation.
Moving type-checks to compilation makes them more robust, but at the cost of the power you gain from dynamic language features. It’s a tradeoff, not a straightforward benefit. There are things you simply cannot do in a statically-typed language.
The real problem here is that you allowed poor coding practices to ship. If you care enough about correctness to switch languages, then you care enough to add systems that enforce this. It’s a false dichotomy.
No the fuck it doesn’t are you kidding me. You are fully able to write incorrect Go code. You still need to write tests in Go.
I would even argue that “if it compiles, it works” is more realistic in Python + mypy than it is in Go:
This is of course assuming that everyone is very disciplined about using types and not abusing
Any, which may be a strong assumption.Passing the compiler’s type check is not sufficient to prove your program is doing what it is supposed to do.
There is one important distinction that I see here, and that burned me several times. If mypy fails, I can ignore it and still run by that python code, only to possibly fail later. If it fails in go, there won’t be anything for me to run.
Nevertheless, I agree with you that there are some classes of errors that could be caught and prevented with mypy.
Kinda funny, it’s the opposite for me: when trying things out the flexibility of bypassing type checks helps, and CI ensures type checking is never ignored in prod.
In find the Go compiler especially annoying when experimenting because it won’t even let me run code with unused variables and imports, they really don’t matter when experimenting.
The core argument for compile time type safety is that sooner or later you always allow poor coding practices to ship. The larger codebase and the longer lived it is the more likely that there are dangers lurking in there for the next refactor that you won’t discover until you hit production. Compile time checks can’t eliminate them totally but they can reduce their frequency considerably. For some people and projects that’s a really valuable attribute.
The argument being made here is that Go’s types are preferable to Python because developers can’t be trusted to test their code. They are literally suggesting that types are a replacement for test coverage.
This is delusional. Types reduce the amount of testing needed, but they will never replace it. If your developers are this unprofessional, I guarantee they will find a way to fuck up in Go as well.
No, I am not. Could you tell me how did you get that impression? I’d be happy to add a correction
So your argument is that Python is a bad choice because
This is a people problem, not a technical one. Using a static type system will not solve this lack of professionalism.
No, he said that enforcing 100% code coverage will piss people off.
Meanwhile, elsewhere in this thread, people are literally arguing that 100% path coverage is necessary and that static types somehow offer the ability to do that.
I am attempting to give them the benefit of the doubt; if they intended to say that type-safety is equivalent to 100% code coverage, then that would be even more nonsensical.
You’re pretty obviously not giving him the benefit of the doubt. The “benefit of the doubt” reading is “getting and enforcing 100% code coverage is harder than using static types in conjunction with less-than-100% code coverage.”
Nobody anywhere is suggesting that 100% code coverage is reasonable. This is a dumb strawman about testing. I’m pretty sure that you know this.
When somebody writes a blog post that tells me the things I have spent the better part of a decade-plus career doing is impossible, I am gonna call bullshit.
I have literally gotten in arguments with people who definitely think that 100% code coverage is both reasonable and necessary. Perhaps it’s not actually a strawman but something they have encountered in their own career?
Type checks that happen at runtime are not useful. In fact, they shouldn’t even be called types at that point. Types are for determining if static code terms are valid or not. And to even say this, it sounds like you didn’t read the article:
Dynamic types don’t help with that. And that’s not a “silly” argument. Do you realize that’s insulting?
The fact that you have a personal definition of types that requires static checks does not mean that all of computer science agrees with you.
If you are working with a dynamic language, dynamic type checks are not only useful they are absolutely necessary.
The question is when a check gives you the most value. You can say an exception is a unit test that runs in production, but I prefer to run my tests as early as possible, with reasonable effort. For the class of invariances that can conveniently be expressed and checked using a type system, that’s what I prefer. For those that can be caught by unit tests, that’s the best. For those that require integration tests… etc. All the way to load tests using realistic fake traffic, if that’s needed.
All depends on what you mean by “works”. Of course, in any moderately complex system, there are an infinite number of intended behaviors that can’t be expressed through a type system (or unit tests). The closest thing I’m aware of is Elm, where the combination of a good (and rather simple) type system and very limited runtime capabilities make it so that if it compiles, it’s extremely rare that there are runtime exceptions. It brings me a lot of joy to be able to skip out on the part of the development cycle where I finish editing some code and wait to see if the program crashes or not. It’s great to just skip ahead to seeing if the higher-order behaviors are as I expected or not. It really does save me time and, most importantly, helps me keep mental focus and enthusiasm.
This is perfectly reasonable, but I will point out that I follow this approach in dynamic languages as well. Dynamic languages are capable of exactly the same constraints (in fact, they can be even more sophisticated), but you have to go about it differently.
Type-checking at compile time comes at a cost, the robustness of the checks means you are giving up flexibility to do certain things. This may be a worthwhile tradeoff for your situation — but it should be made intentionally.
I want to push back against this notion that you need static types to achieve this level of correctness. You do not. Don’t blame your tools for unprofessionalism.
Could you show me some affect of a program which is possible to produce with dynamically type checked tooling but not statically typed tooling?
I’d argue that the industrial benefits of static typing are so obvious and accessible at this point in time that it’s unprofessional to operate under the assumption that “comprehensive” testing can replace the many roles of static tooling.
Here’s one example: I practice repl-driven development on a daily basis. IDE type hints can’t achieve this level of real-time feedback loop.
https://www.idris-lang.org/ (in action, ditto)
Many other examples of statically typed languages with REPL’s exist but this one is my favorite as it’s almost designed around progressive interactions with the REPL.
But the Python type hints are extremely flexible and gives you lots of power to take advantage of all the great dynamic language features of Python.
They are also not an all-in-or-nothing kind of thing. If you have code that needs to be super dynamic and creative and type hints get in the way then you can just as easily decide to not use them there.
(Or - in many cases you can usually create an “outer” layer of typed API while the inside is all the Python magic you want)
I believe both our arguments are perfectly compatible here. I don’t really consider Python’s type-hints to be in the same category as Go’s type checking, I am attempting to speak more broadly about dynamic vs static type systems and not get bogged down in a specific implementation.
Well, there’s your first problem. Yes, if you compare code that you wrote in a dynamically typed language, when you were inexperienced and in a hurry, with code that you wrote in a statically typed language, when you were more experienced and had more people collaborating with you, guess which one will be better. It’s not an indictment of dynamic languages though, but rather of your methodology. I know this is not a scientific article but you gonna make arguments, they should be good.
Cool, what does this function do?
Dynamic languages have pros and cons like anything else. I might even agree that they might have more cons than certain statically typed languages. But there’s no technology that can prevent writing bad code.
Without good processes and experience, your code is gonna suck in any language. It might be marginally better with types, maybe, but even that is not a given.
Unless your hypothetical language has immutable parameters, the only thing I can assume there is that
fbmight be modified byfrobnigate()(less sojands, because most languages these days are pass-by-value; otherwise if the language is pass-by-reference, then I’d be worried about all the parameters). But aside from that, I’d have to look at the code (most likely documentation is sparse).That’s not an hypothetical language, it’s go. Just the type signature.
The point of the example is that types don’t magically make code easier to understand later.
Actually, if sufficiently polymorphic, they do:
I can tell you just from the type signature that any element in the result list must have come from applying the function argument to one of the elements of the input list.
Would it be as easy to tell it if the signature looked like this?
The idea of shipping a massive system to users without strict types seems insane to me. How do you know your tests cover every viable path through your code? How on earth do you maintain a legacy codebase with 0 types and 0 documentation? How do you ensure any code path is valid if values can be and type at any time? I wish these were strawmen, but I keep running into abandoned (and fresh) Python apps that have these problems.
I hate how discourse for static vs dynamic typing pits the two extremes against each other. TypeScript and other gradual type systems are a great middle ground: gradually add types where your team sees fit. You could completely ignore types while developing and make adding type signatures a blocker for merging.
This is one reason I ran away screaming from webdev: everyone wanted to do exactly this, complained that test times were insane (because they didn’t want to take the time to design for testability), and complained that upgrading their framework was painful (because they coupled it directly to the framework). Doing otherwise was taboo, something something not delivering business value.
They were so intent on “disruption” they didn’t have time to stop and think about what would enable them to do that as effectively as possible.
Keep in mind that this means very different things in dynamically-typed languages than it does in statically-typed languages. Statically-typed languages basically force you into Enterprise Code™ patterns because of the inflexibility of the language – you have to do dependency injection and inversion of control and five dozen “decoupled” layers of abstraction between you and the world, because the language itself is incapable of letting you do anything else.
But in dynamically-typed languages, many of those patterns stop being relevant because the language-level constraints that inspired the patterns are not present. Consider, for example, the
unittest.mockmodule in the Python standard library: a gigantic swathe of “testability” patterns from statically-typed languages go out the window because of this one tool, which in turn can only exist because of high runtime dynamism in the language itself.Meanwhile, I’m not sure why “test times were insane” is a counterargument to dynamic typing, considering how many times I hear that people are desperate for compiler speedups because of the multi-hour build times for their supposedly well-engineered software.
Another middle ground is Crystal which I’m using more and more recently where I’d use Python otherwise. There’s enough type inference that I add a type signature maybe once every 50+ lines. Meanwhile all the rest of the code is automagically typed and the compiler will tell me where I made mistakes.
(Most of the typing necessary is for collections, where… yeah, they come extremely useful and stop accidents)
By writing unit tests, the same way you’d do it in a statically-typed language. I’ve never seen a statically-typed language that automatically derives prose documentation and 100% path coverage from the type declarations, so I’m not sure why static typing is supposed to affect this.
Well, I hate to be the bearer of bad news, but legacy projects in any language tend not to come with useful documentation. And given how easy it is to do things like accept a
HashMapof “arguments” in Java and Java-like languages, or to just declare everything asinterfacein Go, I don’t really see how the type signatures are expected to be a panacea. And if you think there aren’t significant real-world real codebases doing those things, you’ve got another think coming – static types don’t and didn’t do anything to prevent that.You test the paths you support. If a client of your code does something wrong, their tests will break. Not because you wrote tests to exhaustively type-check your arguments, but because passing the wrong type of argument tends to throw a runtime – and unit testing is runtime! – error anyway.
It’s possible to write terrible undocumented abandonware in any language.
It is interesting that most comments here do not say anything about developer experience. Type hints in Python don’t just prove your code is more correct. They also power excellent tools like PyLance or python-lsp-server. In my opinion there is nothing more productive than hitting “.” and seeing what properties and methods are on a thing you are going to access. I can make decisions in a matter of seconds instead of vague guess work or assumptions or having to grep through code to find whatever it is that I am working with.
Type hints improve the developer experience 10x. At least. Whether you are in a team of 50 or a solo developer. Many major libraries now have type hints - hacking Python was already productive, but it skyrockets with type hints and a good IDE to show you choices/details and even documentations right there.
I’ve had this same experience with Python. FWIW Python’s standard library has been better than JavaScript’s so you’d be less likely to have
cannot read property 'foo' of undefinedsince dictionaries have agetmethod andNoneis only value to representnullorundefined. I think this gave Python an advantage in the earlier days of the dynamic language showdown between NodeJS and Python. At least in my experience at work there would be way more bugs in NodeJS projects than in Python ones even without types.Today, I don’t feel confident writing code in Python, even with types. They feel clunky to me with their syntax. It seems that types are not even that widely adopted in Python libraries. TypeScript on the other hand has become way more popular within the JS community and is a nicer syntax for writing JavaScript code with types. Even though I prefer Python as a language, TypeScript’s type system feels better to me than Python’s. TypeScript also has libraries like fp-ts and zod which make it easy to parse and validate input and keep type errors at the edge of your program.
Unfortunately, to me at least, Python feels like a dead language. I no longer want to use it, but I do miss a lot of things it has (comprehensions, dataclasses, SQLAlchemy & alembic, pytest, etc.).
Years ago, I started thinking about writing a blog post entitled “How Python became my third favorite language.” The idea was to list all the things that had made it my favorite language back in ~2006 and then write about how Go and JavaScript had usurped Python by surpassing it in those areas. I’ve never written the post in part because of laziness and in part because it’s a bit of a downer, but what you’re saying about Node/TypeScript is definitely one element. I don’t even use TypeScript myself, but because other people do, I get automatic type inference to the frameworks I call into. It has been a while since I tried to use static types in Python, but my last attempt bailed pretty quickly because some popular library wasn’t compatible with what I was trying to do, so I just gave up.
I wonder how much impact the packaging system has on this productivity.
I think a huge amount. IMO NodeJS (and then Go) stole a lot of marketshare of new developers away from Python because setting up Python correctly (with virtualenv) is such a nightmare, especially on Macs. It’s a lot more simple to just install the runtime and then install the packages for a project locally instead of some path relative to where the language is installed which is is stored in a shell env var.
During development itself, not much. The packaging issues affect the initial project setup and big migrations. Otherwise, you can mostly ignore it and just add packages like you always do and update them like you always do. Put the commands in the readme if needed.
So I’d go with some impact in general, and almost none for “this productivity” as described in the post.
I think it’s on point that the main problem is with refactoring. Adding new code is not hard, because it requires testing just the freshly added code, which can be done manually. But refactoring can affect many places all over the codebase, and requires finding them all.
Use pyright and flake8 and you’ll have a similar experience to coding on Typescript with a linter.
I believe this is because the current set of dynamically typed languages we have are at a local maximum, where the type information is perfectly clear and fully introspectable at runtime (even more so than statically-typed, compiled languages), but the language has no tools to introspect or take advantage of these types, because the development context is outside of the runtime where such types are unavailable. And for that matter, I don’t think it’s types that we actually wish for either; what we’re looking for is interfaces that allow us to ask questions to objects and receive responses, and concrete types happen to be one common use of such a construct.
From these two points, I envision the next step in dynamically typed programming to be a system where the development workspace executes in the same context of the program being modified. The workspace allows you to modify and extend a program while it’s running, and the type information that’s available at runtime guides the programmer in creating interface definitions for both documentation and as guides while editing.
Obviously, such a system needs to have proper isolation mechanisms in order to prevent the workspace from being affected by any errors that occur within the program. A process abstraction would be in order here, as well as a way to prevent untrusted programs from doing things that the user didn’t intend (a capability based system sounds nice).
yes! So… Common Lisp and its image-based development?
Either Lisp or Smalltalk (because the two paradigms are isometric). I’m working on a version of the Self programming language that I want to eventually get to the point I’ve described.
[Comment removed by author]
TL;DR: Typing is a useful tool making software cheap to maintain in the long term, which the author now needs.
Strong typing has a small cost for the individual developer and provides a benefit in helping coding between authors or across long time spans. The author has changed jobs from a goal of cheap in the short term to cheap in the long term and now extols typed languages.
As always when the type wars come up, I want to remind people that the evidence for statically typed languages improving software quality is limited and ambiguous.
The whole ‘i have unit tests to cover me for typing and refactoring’ was always a bad excuse. It was for Ruby ,where I think it started to be a meme, and it is for Python.
It was strongly believed and advocated by many at the time. It’s actually shocking to see such a large majority now on the side of strong typing. (I was always on the strong typing side, so I’m not shocked that strong typing is so powerful … I’m shocked at how large a percentage of developers have come to that conclusion).
Apparently, this won’t be like the spaces/tabs and emacs/vi(m) religious wars … we might reach mainstream consensus on this one.
I’ve expressed a similar sentiment before. I’ve done most of my work in C, C++, Java and Python. And for much of my career, I’ve felt most productive in C++, due in no small part to compile-time type checking.
Over the past few years, I’ve changed my opinion a little bit. Python’s type system lets me test ideas and figure things out very quickly. And once I have, its type hints give me almost all of the compile time checking I have always leaned on in C++ (and later Java).
The ability to not need to write down the types to try out an idea, but to then be able to write down the types and have them warned/enforced before I ship has proven quite productive and powerful for me. And when I stack on the ability to have pydantic use them too, it feels like a super power. So as a very C++-biased person, I have lots good to say about python’s typing. Which means I disagree with this post in the large, I suppose, even as it does resonate with me.
i love the sentiment, i hate python, but i write a lot of it. so i posts like this makes me feel people are working on improving my life UwU
Switch languages. Python is poisoning your mind.
For many of us, Python is an integral part of our career. I have yet to work somewhere that did not have a Django or Flask application.
I’m sorry for your loss.
I kind of feel the same way even though I’ve written a very small amount of Go. While I do try to add type hints wherever possible to aid my future self, it’s nowhere near as close to the level of helpfulness as, for example, in Go where VS Code just says flat out that doing X in place Y would definitely be incorrect for Z to still be true. This is can be alleviated somewhat with Pylance’s strict mode, if I’m not mistaken, which will emulate the same experience as in Go, but if you haven’t gone the strict type hinting route from the very start then it’s just going to be a nuisance with you trying to retroactively type hint properly all the old things in order for the new additions to be valid.