Skip to content

Instantly share code, notes, and snippets.

@LakshyAAAgrawal
Last active February 20, 2024 12:36
Show Gist options
  • Star 21 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save LakshyAAAgrawal/33eee2d33c4788764087eef1fa67269e to your computer and use it in GitHub Desktop.
Save LakshyAAAgrawal/33eee2d33c4788764087eef1fa67269e to your computer and use it in GitHub Desktop.
Pytranslate - Maxima to Python Translator

Pytranslate - Maxima to Python Translator

Introduction

Package pytranslate, introduced through merge #15 and #17 as a loadable package in Maxima provides Maxima to Python translation functionality. It was developed under the mentorship of Dimiter Prodanov and Robert Dodier as an INCF project during Google Summer of Code 2019 by Lakshya A Agrawal of Indraprastha Institute of Information Technology, Delhi, India.

Python has extensive library support and gaining a stronghold in the field of neuroinformatics. Maxima is an expressible computer algebra system and language. A Maxima to Python Translator will provide a way for implementation of algorithms in Maxima which make use of the extensive libraries in Python. One can write certain parts of algorithms in Maxima and others in Python, and ultimately translate Maxima to Python, integrating both. Further, for a person comfortable in Maxima, it will be easy to share their work with a person unfamiliar with Maxima.

The project proposal for Google Summer of Code is available here.

Pytranslate is written in Common Lisp and Maxima. The source code is present in the Maxima repository.

Usage

Pytranslate can be loaded in a Maxima instance for use, by executing load(pytranslate); The statements are converted to python3 syntax. The file share/pytranslate/pytranslate.py must be imported as from pytranslate import * for all translations to run, as shown in the example below.

Example run

Example:
In maxima:

(%i1) load (pytranslate)$
/* Define an example function in Maxima to calculate square, and call pytranslate */
(%i1) pytranslate(sqr(x):=x^2); 
(%o1) 
/* Translated python3 code */
def sqr(x, v = v):
    v = Stack({}, v)
    v.ins({"x" : x})
    return(f["pow"](v["x"], 2))
f["sqr"] = sqr

(%i3) sqr(5);
(%o3) 25

In python3:

>>> from pytranslate import *
>>> def sqr(x, v = v):
...     v = Stack({}, v)
...     v.ins({"x" : x})
...     return(f["pow"](v["x"], 2))
... 
>>> sqr(5)
25.0

Plotting example:
In maxima:

(%i1) pytranslate('(plot3d(lambda([x, y], sqrt(2-x^2 -y^2)), [x, -1, 1], [y, -1, 1])));
(%o1) f["plot3d"](lambda x, y, v = Stack({}, v): math.sqrt((2 + 
							   (-f["pow"](x, 2)) + 
						           (-f["pow"](y, 2)))), ["x", -1, 1], ["y", -1, 1])

Generated Plot:

3D Plot
(Left - Maxima), (Right - Python)
2D Plots
Example 2D Plots - mwright and cantor functions

Supported Maxima forms

  1. Numbers(including complex numbers)
  2. Assignment operators(:, :=)
  3. Arithmetic operators(+, -, , ^, /, !)
  4. Logical operators(and, or, not)
  5. Relational operators(>, <, >=, <=, !=, =)
  6. Lists, endcons
  7. Arrays
  8. block(mprog) and mprogn
  9. Function and function calls
  10. if-else form
  11. Loops(do, while, unless, for, from, thru, step, next, in)
  12. lambda form
  13. Plots(2D and 3D) using matplotlib, numpy
  14. Integration using Quadpack(mpmath), demonstrated on quadqagi

Files in pytranslate

  • share/pytranslate/pytranslate.lisp - contains all the translation procedures and working logic of pytranslate.
    Introduced as transpiler.lisp in commit [764b7a].
    Renamed to pytranslate.lisp in commit [9481a9]

  • share/pytranslate/rtest_pytranslate.mac - contains the tests for pytranslate. Among many others, the tests include example functions to generate continued fraction representation of a given number, generate the number from it's continued fraction representation, and cantor function.
    Introduced in commit [25f90b]

  • doc/info/pytranslate.texi - contains the documentation for pytranslate usage, working, and method to extend pytranslate.
    Introduced in commit [acef78]

  • share/pytranslate/pytranslate.py - All translated code requires the import of pytranslate.py, which defines several function and variable maps from Maxima to Python. It defines Stack, and variables v and f which are used to hold variable bindings in translated code per scoping rules in Maxima.
    Introduced in commit [f38107]

  • share/pytranslate/cantorr.py - Example translated file - All functions are translated from Maxima containing algorithmic and plotting code.
    Introduced in commit [67a246].

Tests for pytranslate

The tests for pytranslate are present at share/pytranslate/rtest_pytranslate.mac and can be evaluated by executing batch(rtest_pytranslate, test);

Functions in pytranslate

Function: pytranslate (expr, [print-ir])

Translates the expression expr to equivalent python3 statements. Output is printed in the stdout.

Example:

(%i1) pytranslate('(for i:8 step -1 unless i<3 do (print(i))));
(%o1) 
v["i"] = 8
while not((v["i"] < 3)):
    m["print"](v["i"])
    v["i"] = (v["i"] + -1)
del v["i"]

expr is evaluated, and the return value is used for translation. Hence, for statements like assignment, it might be useful to quote the statement:

(%i2) pytranslate(x:20);
(%o2) 
20
(%i3) pytranslate('(x:20));
(%o3) 
v["x"] = 20

Passing the optional parameter (print-ir) to pytranslate as t, will print the internal IR representation of expr and return the translated python3 code.

(%i2) pytranslate( '(plot3d(lambda([x, y], x^2 + y^(-1)), [x, 1, 10], [y, 1, 10])), t);
/* Generated IR */
(body
 (funcall (element-array "m" (string "plot3d"))
  (lambda ((symbol "x")
                       (symbol "y")
                        (op-no-bracket = (symbol "v") (funcall (symbol "stack") (dictionary) (symbol "v"))))
   (op + (funcall (element-array (symbol "m") (string "pow")) (symbol "x") (num 2 0))
               (funcall (element-array (symbol "m") (string "pow")) (symbol "y") (unary-op - (num 1 0)))))
  (struct-list (string "x") (num 1 0) (num 10 0))
  (struct-list (string "y") (num 1 0) (num 10 0)))) 
(%o2) 
f["plot3d"](lambda x, y, v = Stack({}, v): (f["pow"](x, 2) + f["pow"](y, (-1))), ["x", 1, 10], ["y", 1, 10])

Function: show_form (expr)

Displays the internal maxima representation of expr.

Example:

(%i4) show_form(a^b);
((mexpt) $a $b) 
(%o4) a^b

Working of pytranslate

  • The entry point for pytranslate is the function $pytranslate defined in share/pytranslate/pytranslate.lisp.
  • $pytranslate calls the function maxima-to-ir with the Maxima expression as an argument(henceforth referred to as expr).
  • maxima-to-ir determines if expr is atomic or non-atomic(lisp cons form). If atomic, atom-to-ir is called with expr, which returns the IR for the atomic expression.
  • If expr is non-atomic, the function cons-to-ir is called with expr as an argument.
    • cons-to-ir looks for (caar expr), which specifies the type of expr(like addition operator, function call, etc.), in hash-table *maxima-direct-ir-map*. If the type is found, then it appends the IR retrieved from hash-table with the result of lisp call (mapcar #'maxima-to-ir (cdr expr)), which applies maxima-to-ir  function to all the elements present in expr. Effectively, it recursively generates the IR form for all the elements present in expr and appends them to the IR map for the type.
      Example:

      (%i9) show_form(a+b);
      ((mplus) $b $a)
      
      (%i10) pytranslate(a+b, t);
      (body (op + (element-array (symbol "v") (string "b")) (element-array (symbol "v") (string "a"))))
      (%o10) 
      (v["b"] + v["a"])

      Here, operator + with internal maxima representation - (mplus) is present in *maxima-direct-ir-map* and mapped to (op +) to which the result of generating IR for all other elements of the list (a b) - (element-array (symbol "v") (string "b")) (element-array (symbol "v") (string "a")) is appended.

    • If type of expr is not found in direct translation hash-table, then cons-to-ir looks for the type in *maxima-special-ir-map* which returns the function name to handle the translation of expr-type. cons-to-ir then calls the returned function with expr as an argument. Example:

      (%i11) show_form(g(x):=x^2);
      ((mdefine simp) (($g) $x) ((mexpt) $x 2))
      (%i12) pytranslate(g(x):=x^2, t);
      /* Generated IR */
      (body
       (body
        (func-def (symbol "g")
                  ((symbol "x") (op-no-bracket = (symbol "v") (symbol "v")))
                  (body-indented
                      (op-no-bracket = (symbol "v") (funcall (symbol "stack") (dictionary) (symbol "v")))
                      (obj-funcall (symbol "v") (symbol "ins") (dictionary ((string "x") (symbol "x"))))
                      (funcall (symbol "return")
                               (funcall (element-array (symbol "f") (string "pow"))
                                        (element-array (symbol "v") (string "x"))
                                        (num 2 0)))))
        (op-no-bracket = (element-array (symbol "f") (string "g")) (symbol "g"))))  
      (%o12) 
      def g(x, v = v):
          v = Stack({}, v)
          v.ins({"x" : x})
          return(f["pow"](v["x"], 2))
      f["g"] = g

      Here, mdefine, which is the type of expr, is present in *maxima-special-ir-map* which returns func-def-to-ir as handler function for mdefine, which is then called with expr for IR generation.

  • After the generation of IR, the function ir-to-python is called with the generated IR as an argument, which recursively performes code generation.
    • ir-to-python looks for lisp (car ir), i.e., the type of IR, in the hash-table *ir-python-direct-templates*, which maps IR type to function handlers and calls the handler function with IR as an argument.
    • The handler function generates python3 code recursively as a string. It uses a template corresponding to the type of IR and calling ir-to-python on all subexpressions.

Future Scope

Maxima, being a vast language, Pytranslate supports a core subset of its main features. Future work in Pytranslate would extend it to support a broader base. This work can include:

  1. Defining procedures for translation of unique forms in Maxima to defined IR by modification of *maxima-special-ir-map*
  2. Extending IR definition to support more extensive python syntax by adding to *ir-python-direct-templates*
  3. Implementation of Maxima functions in Python and adding to the functions mapping in pytranslate.py

Maxima function whose translation isn't currently defined can easily be translated by defining an equivalent function in python and adding it to pytranslate.py.

Work on symbolic manipulation was not performed during Google Summer of Code and can be looked into for extending Pytranslate, as described.

During the initial project development, direct variable access in Python was being used, which however resulted in issues with maxima variable scoping in python. Currently, the class Stack, which defines a dictionary-like access to maxima variables in python, is used. Benchmarking shows a 2-7x slow down in speed compared to direct variable access. An alternative mechanism for variable access would be a step towards faster and more readable translation.

Further, Pytranslate can be modified to write the generated python output to a python module, and perform integrated Maxima and Python tests.

Student Developer Information

Name Lakshya A Agrawal
GitHub LakshyAAAgrawal
Matrix LakshyAAAgrawal
LinkedIn LakshyAAAgrawal
Twitter LakshyAAAgrawal
Mastodon @LakshyAAAgrawal@fosstodon.org
E-Mail lakshya.aagrawal@gmail.com

Commits

Commit Description
[304a29] Merge branch 'translation' including documentation - Latest commit as part of Google Summer of Code development
[defa4a] Documentation on working and extension of pytranslate
[5d3d1c] Merge branch 'translation' with pytranslate, Maxima to Python Translator
[d4781a] Made minor correction in function variable mapping name
[4209df] Preparation for final merging into master - Documentation file pytranslate.texi
[67a246] Add support for plotting 2D and 3D functions. Add cantorr.py as example translation.
[45fa67] Make changes for multiple function plots, support for integration using quadpack from mpmath
[c51c95] Add support for translation of code plotting 2D functions, using matplotlib and numpy. Make changes for adding all functions to functions dictionary after declaration
[ed1d8b] Make changes for initialization of HierarchicalDict during function execution rather than declaration. Introduce cantorr2 as testcase, evaluates cantor function
[dc68eb] Make changes to support handling of array indexes, fixes bug
[f1626c] Make changes for appropriate handling of if-else within function calls, and as statements. Add numberp, listp, and length functions in pytranslate.py
[3b481c] Add support for both types of control structures: if and conditionals
[507fbb] Merge branch 'translation_dictionary' into translation, introducing HierarchicalDict and pytranslate with corresponding changes for appropriate scoping of maxima variables in python
[129b26] Finished code changes for use of HierarchicalDict in pytranslate
[075bad] Introduce HierarchicalDict handling maxima variable scoping in python and modify translation of mprogn/mprog. Introduce functions 'num' and 'denom' in pytranslate.py
[f38107] Introduce pytranslate.py, with mappings for maxima functions and variables in python. Add support for python dictionaries and object function calls. Introduced function preprocess for testing.
[c9fd8e] Add support for mreturn() funcall and a testcase for endcons
[0fef6a] Add support for endons, floor, fix, sqrt, uninitialized for loop. Add option print-ir to pytranslate. Bug fixes.
[4518e0] Merge branch 'translation' to 'master'
[acef78] Add English texinfo documentation for pytranslate, Maxima to Python Translator.
[808eba] Made minor changes in main Maxima README and Documentation fixing references about adding directory to share
[8d1816] Add test for multiple expressions in lambda
[c957a1] Add support for variants of for loop
[a5079b] Add tests for forms generating random function names with gensym(block, mprogn, mprog) by use of regex
[aef408] Add support for "for loop through list" -> for var in list do exp. mprogn within for loop is converted to IR block-indented.
[92ef66] Make changes for return of last expression in function definition, pass function arguments to generated block function
[0051f4] Add support for multiple expressions in lambda forms along with optional variables
[9d991a] Add support for Lambda forms along with tests
[b3ef84] Make changes for appropriate conversion of symbol names, preserving symbol name case
[25f90b] Add test file rtest_pytranslate.mac for testing Maxima to Python Translator
[b9f2b7] Fix variable reference error
[2a3c40] Changes based on code review from community
[80d851] Add nformat preprocessing prior to translation
[9481a9] Rename from transpiler to pytranslate
[516107] Add support for conversion of IR to Python code - introduce function ir-to-python
[c567f6] Fixed mfactorial bug, work on conversion of mprogn, mprog and if-else expressions
[dac397] Modify function handling conversion of Maxima functions to IR to handle variable argument lists
[466ad6] Add support for conversion to IR of logic operators, relational operators and array references.
[52a772] Add support for converting block(progn) statements, and array definitions to IR.
[9677c4] Add maxima-to-ir support for function definition, lists, and assignment operator. A detailed conversion chart is present in the file share/transpiler/maxima-to-ir.html
[31d48d] Started work on maxima-to-ir function.
[764b7a] Initial Commit

Other contributions

I also worked on implementation of the Quine Mccluskey algorithm for boolean simplification under the name of Ksimplifier in share/logic package. Ksimplifier was introduced through merge #10. Bug fixes inroduced through merge #17 based on inputs from Stavros Macrakis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment