Feasibility, Use Cases, and Limitations of Pyodide
Coming into an internship you never really know what to expect from your summer. What I definitely didn’t expect was that my whole summer would center around one library – Pyodide – and that it would leave such an impression on my team and organization.
What is Pyodide?
Pyodide is an open-source library that was started at Mozilla as part of the Iodide project. Pyodide allows a user to run Python in a web browser through having the Python interpreter compiled to Web Assembly (WASM). It comes with 75 packages already in the base library, most of which are part of the scientific stack such as NumPy and Pandas. Other packages can be installed if they are pure python wheels from PyPI. Check out the Pyodide library here and the docs here.
With my project being very open-ended, the question I wanted to investigate was how can my team, the Python Tools for VS Code team, use Pyodide? The idea of running Python in a web browser seems amazing, but how easy is it to work with and what can we actually prototype?
What use cases can we see Pyodide filling?
Since Python is a great language for beginners, we always keep beginner programmers and students top of mind when we think about who our users are and what they need. We have found that beginner users struggle to install and configure Python and these steps are a barrier during their getting started experience. Additionally, we have seen that some schools who restrict network access on school computers accidentally block students from downloading Python onto a school computer. Both of these problems could be fixed with the design of an extension, accessible through the VS Code marketplace, which would allow a user to run their Python code via Pyodide on the web. This was a very compelling use case and so I began exploring it as an option and I learned so much about Pyodide along the way. We were able to solve the problems with an extension as you can see in the demo below but this involved some workarounds that made it not production worthy. Here are my biggest takeaways and where I see this library going.
How Pyodide Works and Its Limitations
VS Code is built on top of Electron which is an open source technology used to build native Node applications. During the process of trying to write a Pyodide extension for VS Code I discovered that Pyodide did not have Node support; it required a window object, which exists if Pyodide is run in a web worker or in a browser but does not exist in Node. The great news is a fix for this issue has just been released and so our original idea can now be implemented without work-arounds.
Three other important considerations for Pyodide, that might be limitations based on the use case, are: restrictions on importing 3rd-party code, printing/returning script results, and restrictions on multifile programs. The way that Pyodide works is that you import the Pyodide script via a CDN or bundle the library along with your application, although this will increase your project size by 6.4 MB for just the CPython standard library or 172.955 MB for the compressed Pyodide library with the 75 pre-installed packages. Since this is a web-based library, the rest of your application will be written in JavaScript/TypeScript such as a web worker, a web application, or a Node application which can all contain Pyodide. After importing Pyodide you can use a few simple but powerful commands to do everything else you would need with Pyodide. You simply run pyodide.runPython(string of code) and pass in a string of Python code that you want to run. Since Python rules apply, this should be a valid Python script. From there, Pyodide runs the code — there is an async command option as well — and prints any print commands to the console and returns the final return value. Due to this design, the location where Pyodide prints is based on the context in which it is called and only the final return value is captured by the main JavaScript/TypeScript script which is running Pyodide. The second limitation of this design is that Pyodide cannot handle multifile programs because the program is passed in as a string. Unrelated to these two limitations is the design of the package import system: Pyodide bundles 75 packages and additional packages can be imported if they are pure Python wheels hosted online, either on PyPI or another URL.
Advantages of Pyodide
Given these concerns, are the advantages worth the potential limitations? I say yes (although my team makes no promises about the future 😄). I think Pyodide brings a whole new set of opportunities to Python that are poised to become very impactful to all different types of developers. The size of Pyodide is a bit big but it is manageable, and the library has great performance speeds that remain constant across Chrome and Edge during our light testing. Since the project is open-source it is simple to add new supported libraries to Pyodide’s current list of 75 packages, and the library in general continues to be actively maintained by a few core contributors. The activity on the Pyodide repo means that issues are being answered quickly and new features such as Node support are being added consistently to the project. Pyodide is continuously improving, and the limitations right now should not be viewed as static.
Additional Examples and Future Applications
The use case described in this article is just one compelling example of how Pyodide can help users and developers alike. My coworker Jonathan Carter demoed another example on his Twitter which is the extension CodeSwing that he built and maintains which allows users to create/view interactive playgrounds within VS Code and does not require compilation, instead relying on Pyodide to run Python code. Additionally, Pyodide is being explored as a way to run Python tooling in VS Code. During my internship I worked on getting a lightweight intelisense extension for Python working which used Jedi run via Pyodide and was very straightforward since Jedi is one of the packages included in the Pyodide library. Additionally my coworker Joyce Er demoed a personal project on her Twitter that uses Pyodide to run Python code in a notebook and is backed by JupyterLite and uses the VS Code API. Just in the Microsoft Developer Division there are already numerous use cases which demonstrate the power of Pyodide such as running code, Python tooling, interactive coding with VS Code, and Notebooks which all show how useful Pyodide can be if other organizations and individuals start to incorporate it into their projects. Python is such a useful language to developers because it brings an ecosystem of useful packages and Pyodide transfers this power to the customer. Pyodide could solve the problems presented by creating a great Python development experience in new products in and outside of Microsoft and I expect continued growth in this library as other use cases are discovered.
Light
Dark
0 comments