Update to python 3

Parent Previous Next

Migrating old Python 2 scripts to Python 3

The original Python API of EnSight has been built on top of Python 2.7.1, and during the years it has been updated to Python 2.7.11. These updates have made it possible to keep the libraries reasonably up to date, allowing one to take full advantage of newer capabilities and include security patches.

In order to keep being able to update libraries, it is necessary to update the version of Python again. Unfortunately, Python 2.7 will reach end of life in 2020, and Python 3.X has not been designed to be fully backwards compatible with Python 2.X. Therefore, this upgrade will require some extra work to make sure your scripts can still run on newer versions of EnSight.

There are a lot of resources on the web that discuss this change and suggest solutions and help. In this document, we will try to provide some additional help and examples taken from already existing scripts for EnSight that have already been migrated. For more detailed discussions, please refer to the official Python documentation here. Where conflicts arise, please consider the official Python documentation to be the correct one.

If you are interested in knowing why Python 3 hasn't been designed to be backwards compatible, you can find multiple resources online - we suggest the blog posts here and here, and the presentation at this link.

For a useful cheat sheet of how to implement the changes, give a look at this link.

Handling the update: Making scripts forward- and backward-compatible

There are two ways to handle a transition to something that isn't compatible. The first is to wait until EnSight is released with Python 3 as its Python API, and convert all the scripts to work in Python 3 only. The second is to make all the scripts compatible with both Python 2 and Python 3. Since most users will work in an environment where they will be running multiple versions of EnSight (and therefore, potentially both Python 2 and Python 3), we suggest using the second approach. Fortunately, most of the changes that are needed will make the code compatible with both versions of Python. For the few instances where this is not possible, we would suggest refactoring the code.

Timeline

EnSight is currently running Python 3.  Before that transition, EnSight 2019 R2 and EnSight 2019 R3 were released with user-visible Python scripts that have been converted to work in both versions of Python and the documentation was updated to reflect this as well. EnSight 2020 R1 was the first version to contain Python 3. You can go into the doc/ directory in your local installation of EnSight 2019 R2 / EnSight 2019 R3 [ location: $installation_directory/ensight19X/doc ] and you will find a file called python_2_scripts.zip. If you unzip it, it contains a few Python scripts from old versions (compatible only with Python 2.7) that you can use to compare with the current version of the scripts available in your installation, to see what changes have been performed to make the scripts compatible with both Python 2 and Python 3. More information in the README.txt file in the zipped directory.

Conversion examples

Here a list of the conversions that have been performed on the user-visible scripts, to make them compatible with both Python 2 and Python 3. Note that the goal of this conversion was not only to make the script run in Python 3, but also still run and give the same result in Python 2. For this reason, sometimes instead of adopting a Python 3-only solution, an hybrid has been implemented (see the print statement section). You can use this list as a reference for the type of changes you might need to make to your scripts.

Print Statement:

The print statement syntax has been modified. In its easier form, simply add parenthesis:

Python 2:

print "my statement"


Python 2&3:

print("my statement")


If there are multiple arguments for the print statement, then use the .format() syntax:

Python 2:

my_variable = 1.0

print "my var value:", my_variable


Python 2&3:

my_variable = 1.0

print("my var value: {}".format(my_variable))


Note that the Python 3 syntax would have allowed you to simply use:

my_variable = 1.0

print("my var value:", my_variable)


But the same syntax in Python 2 would return a tuple instead of a simple string. Therefore, this simpler syntax would return:

Python 2 output:

('my var value:', 1.0)


Python 3 output:

my var value: 1.0


To avoid having a different output based on the version of Python you are using, we suggest implementing directly the syntax with .format(). Note that the format syntax allows full control over the string formatting. For a reference tailored towards the Python 2 -> Python 3 transition, please refer to this page.

Note also that in Python 2 it was possible to use the print statement alone to add an extra line (to improve readability of the output). This isn't possible in Python 3, since print is a function and not a statement. Therefore, apply the following change:

Python 2:

print


Python 2&3:

print("")

Exceptions:

The exception statement syntax has been modified. When using "except", change the comma into as "as":

Python 2:

try:

       my_test()

except Exception, e:

       print "Error", e


Python 2&3:

try:

       my_test()

except Exception as e:

       print("Error: {}".format(e))


When raising an exception, consider that raise Exception is not a statement any more, requiring parenthesis:

Python 2:

raise Exception, self.last_error()


Python 2&3:

raise Exception(self.last_error())

Execute:

The syntax for the execute command is changed. Once again, execute has gone from being a statement to being a function, so arguments need to be passed in parentheses:

Python 2:

f = open("myfile")

d = f.read()

exec d


Python 2&3:

f = open("myfile")

d = f.read()

exec(d)


If you want to specify the namespace of the function to execute, then pass it as an argument of exec:

Python 2:

mycode = "var1 = 1\nvar2 = 2\nmy_result=va1+var2"

codeObj = compile(mycode, 'my_result_str', 'exec')

ns = {}

exec mycode in ns


Python 2&3:

mycode = "var1 = 1\nvar2 = 2\nmy_result=va1+var2"

codeObj = compile(mycode, 'my_result_str', 'exec')

ns = {}

exec(mycode, ns)

Unsupported methods:

A few methods and syntax statements have been completely dropped in Python 3. This implies that the code needs to be re-written in order to avoid using these methods/syntaxes at all. Here a list of the ones that were used in EnSight's tools:


has_key() method for dictionaries

Python 2:

mydic = {}

mydic['one'] = 1

if mydic.has_key('one'):

       print "Correct!"


Python 2&3:

mydic = {}

mydic['one'] = 1

if 'one' in mydic:

       print("Correct!")


Note: be careful if you use the new "in" notation in logic statements that contain multiple comparisons, since the order in which they are resolved might differ with the new syntax. For example, the Python 2 statement:


if mydict.has_key('one') == False:


does not correspond to:


if 'one' in mydic == False:


since this new syntax might do first the comparison "mydic == False" and then do the " 'one' in ...." part. If you want to make sure the logic is still the same, apply parenthesis:


if ('one' in mydic) == False:


<> syntax

Python 2:

a = 1

b = 2

if a <> b:

       print "Correct!"


Python 2&3:

a = 1

b = 2

if a != b:

       print("Correct!")


Long integers

In Python 2, integers can be 32bit (int, the default) or 64bit (long). In Python 3, all integers are 64bit (and still refers to them as int), making the long definition redundant and potentially introducing a type conflict.  Trying to use the "L" syntax to define a long integer in Python 3 will actually return an error. Unfortunately, this means there is no easy way to create an 64bit integer while specifying that it must be at the same time long if running on Python 2 and an int if running on Python 3.

Python 2:

a = 10L


Python 3:

a = 10


Tuple as function arguments

Python 3 doesn't allow any more to simply pass tuples as arguments into functions. For a discussion on the reason why, see here.

Python 2:

def my_function( (a,b), c):

       pass


Python 2&3:

def my_function( a_b, c):

       a, b = a_b

       pass


xrange

Python 3 doesn't support xrange() any more. Use range() instead.

Python 2:

a = xrange(1, 10000)


Python 2&3:

a = range(1, 10000)


__cmp__() and comparison methods

In Python 3, the object methods used to compare objects changed. The details can be seen here.  Python 2 used a single method __cmp__().  Python 3 uses 6 methods: __eq__(), __ne__(), __lt__(), __le__(), __gt__(), __ge__().


Python 2:

class foo:

    def __init__(self, v):

        self._v = v

    def __cmp__(self, other):

        if self._v < other:

            return -1

        if self._v > other:

            return 1

        return 0


Python 2&3:

class foo:

    def __init__(self, v):

        self._v = v

    def __cmp__(self, other):

        if self._v < other:

            return -1

        if self._v > other:

            return 1

        return 0

    def __eq__(self, other):

        return self._v == other

    def __ne__(self, other):

        return not self._v == other

    def __lt__(self, other):

        return self._v < other

    def __gt__(self, other):

        return self._v > other

    def __le__(self, other):

        return (self._v < other) or (self._v == other)

    def __ge__(self, other):

        return (self._v > other) or (self._v == other)


Note: if you only need Python 3 compatibility, the __cmp__() method is not needed.