ImportError: cannot import name AstroPi

[butchec]

I can no longer run any programs that imports the AstroPi

After working for a few days following tutorials and sample codes, I was testing the set_pixel function to use the 8x8 LED grid as a two line temperature and humidity display.

I was running through testing the output of a two digit number on the AstroPi while not affecting the bottom lone of text and the Pi appeared to 'lockup'. After a reboot, everytime I run a program that imports the AstroPi module I get the same error over and over that I can no longer import the AstroPi module.

I have tried reinstalling the AstroPi software and I have followed every version of re-installing the software from this forum and Google searches. My next step is to wipe the SD card and reinstall the Pi from scratch.

Has anyone else seen this or found a fix?

I have also removed the AstroPi board, checked for damage and re-installed the Board, but this had no effect.

[Davespice]

Hi! There are two ways you can do the import.
Like this:

Code: Select all

from astro_pi import AstroPi
ap = AstroPi()

Or like this:

Code: Select all

import astro_pi
ap = astro_pi.AstroPi()

The following code will not work because the module is named astro_pi (with an underscore).

Code: Select all

import AstroPi

If the two methods above do not work I would suggest to try them in both Python 2 and 3 and see if one of them still works. If neither work then I would, as you suggest, wipe your SD card and reload it. Perhaps you're suffering from some SD card corruption.

[butchec]

Hi,

Thank you for your help. I can see from the code you have listed that I was doing it the correct way for the code I was running.

I also experienced the issues with the examples, and so far none of the examples work.

Please see below the response I get when I try and run any program that uses the AstroPi; I get the same error whether the AstroPi is installed onto the Pi or not.

Code: Select all

Traceback (most recent call last):
  File "numbers.py", line 3, in <module>
    from astro_pi import AstroPi
  File "/usr/local/lib/python2.7/dist-packages/astro_pi-1.1.4-py2.7.egg/astro_pi/__init__.py", line 2, in <module>
    from .astro_pi import AstroPi
  File "/usr/local/lib/python2.7/dist-packages/astro_pi-1.1.4-py2.7.egg/astro_pi/astro_pi.py", line 7, in <module>
    import numpy as np
  File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 170, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 8, in <module>
    from .type_check import *
  File "/usr/local/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 11, in <module>
    import numpy.core.numeric as _nx
  File "/usr/local/lib/python2.7/dist-packages/numpy/core/__init__.py", line 9, in <module>
    from . import numerictypes as nt
  File "/usr/local/lib/python2.7/dist-packages/numpy/core/numerictypes.py", line 101, in <module>
    import numbers
  File "/home/pi/numbers.py", line 3, in <module>
    from astro_pi import AstroPi
ImportError: cannot import name AstroPi

So far I have also tried the following:

- Wiped the SD card and reinstalled everything including all updates possible from RaspberryPI and AstroPi
- Installed the AstroPi onto another B+ model
- Used the AstroPi SD card that came with the pack.
- Run various scripts that test individual sensors built into the AstroPi

None of the fixes or trick work and at no point have I managed to get the AstroPi working, including running the examples created by AstroPi.

From my testing and fault finding I would guess that the AstroPi board is dead as it does not respond to any commands when tested on two separate Raspberry Pi (B+ and a B2) with fresh installs on known good Raspberry Pi's

Does anyone know where I can return the AstroPi for replacement?

[ben_nuttall]

Hi

Am I right in saying the file you're trying to run is called "numbers.py"?

It looks like numpy (a numerical library Astro Pi uses) imports a file called numbers.py and it's confusing it with your own numbers.py

Can you try renaming the file and running again?

Also try just entering "python" in a terminal and entering "from astro_pi import AstroPi" and see if that works ok. Again, make sure there's no "numbers.py" in the folder you start from.

[butchec]

Hi Ben,

You are right, the program(s) I am running all have the word number or numbers in their title. However I am surprised that this would stop all other programs from running including the examples produced for the AstroPi.

I have renamed all of the files and removed any trace of the original files from the Pi. To my surprise the AstroPi sprang back to life.

I am now able to run all of my programs written and the examples that come with the software.

I am amazed that something so simple would have crippled the AstroPi. I testing this with the other RaspberryPi I have and removing the 'number(s)' files has fixed it on that system too.

Thank you for your help.

[ben_nuttall]

It's because Python's import hierarchy starts with files in the current folder before looking in the installed modules.

It is odd, though. I wonder if numpy should be using relative imports "import .number" rather than "import number"?

[DougieLawson]

It's because Python's import hierarchy starts with files in the current folder before looking in the installed modules.

How stupid is that? What an awful design. The current directory should be the last thing searched (unless you deliberately add it at the front of the search path).

[jojopi]

How stupid is that? What an awful design. The current directory should be the last thing searched

Strictly speaking, it is the directory that the main script lives in, rather than necessarily the current directory. Trusting the current directory would be a bad thing.

If you put a module in the same directory as a script, you almost certainly did want the script to use that module, rather than one of the same name that someone else installed miles away.

If you do want the system module namespace to trump your own, you can sys.path.pop(0).

[waveform80]

It's because Python's import hierarchy starts with files in the current folder before looking in the installed modules.

It is odd, though. I wonder if numpy should be using relative imports "import .number" rather than "import number"?

Indeed - the default Python search path looks something like this (from my Ubuntu machine):

Code: Select all

dave@spider:~/picraft$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/dave/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client', '/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode']

You can see the first entry there is '' which simply means the current path. If you look through the stack trace from the top you can get an idea of what's going on:

• You run something like "python numbers.py" so the Python interpreter starts up and places your module as "__main__" in sys.modules (which is the cache of loaded modules)
• numbers.py executes "from astro_pi import AstroPi". Python first checks whether "astro_pi" is in the sys.modules cache; it isn't so Python searches for the astro_pi package in sys.path and when it finds it, places "astro_pi" in sys.modules
• astro_pi/__init__.py executes "from .astro_pi import AstroPi" which means it wants to import the astro_pi module from within its package (also called astro_pi). Python searches for "astro_pi.astro_pi" in sys.modules, doesn't find it, and so searches for it under the astro_pi package's path (the "." makes this a relative import so it doesn't search the whole of sys.path). It's found and bunged in sys.modules as "astro_pi.astro_pi"
• astro_pi/astro_pi.py executes "import numpy as np". Python searches for "numpy" in sys.modules, doesn't find it, and searches sys.path for it. It's found and added to sys.modules as "numpy"
• numpy/__init__.py executes "from . import add_newdocs". This is another relative import so Python searches sys.modules for "numpy.add_newdocs", doesn't find it, then searches numpy's path for add_newdocs.py, finds it and adds it to sys.modules as "numpy.add_newdocs"
• Dave's fingers get tired from all this typing and I'm sure you've got the idea by now so things continue like this until...
• numpy/core/numerictypes.py attempts to execute "import numbers". Python searches sys.modules for "numbers", doesn't find it, searches sys.path and finds it in the current directory. Python adds it to sys.modules as "numbers". Numpy gets a module it didn't expect containing stuff it doesn't want and things go "bang"!

Now, Ben pointed out numpy ought to be doing a relative import of numbers, like "import .numbers" or "from . import numbers" to ensure that only numpy's path is searched and not the whole sys.path. That would indeed be sensible ... if the numbers module that numpy is looking for was part of numpy itself. Unfortunately it's not. Instead it's looking for the numbers module from the standard library; it's in Python 2 and 3 and it defines the abstract base types for numerals (integral, rational, real, etc.). Because numpy's looking for something outside itself it has no option but to use an absolute import, and thus the problem arises.

So, the solution is simply: don't name your scripts after top level packages in the standard library (and preferably top level third party packages too). Admittedly the standard library of Python is unusually large (the "batteries included" philosophy and all that) so this is harder than in other languages, but sticking with patterns like "my_numbers.py" or "numbers_1.py" tends to remove such problems (naming packages is a tougher problem!).

Now, onto Dougie's criticism and solution: just shove the current directory to the end of the sys.path and all will be well! In the scenario presented above that's absolutely correct. Unfortunately from the point of view of language design, it's just shoved the problem elsewhere...

Consider the case where the user now decides that "numbers" should be a module that their main script calls, so they've got a "main.py" and "numbers.py". They run "python main.py" and the first thing it does is "import numbers". Now Python searches sys.path and finds ... the standard library's numbers module instead of the user's expected module and things go "bang!" again. The core problem is still the same: the user has named a module the same thing as a top-level module/package - the only difference is when things go bang.

There are a number of "proper" solutions to this but for one reason or another none of them are going to happen:

• Ensure everything sits under a namespace unlikely to be duplicated. For example, stick the entire standard library in a "stdlib" package so instead of "import numbers" you'd have to do "import stdlib.numbers" and instead of "import os.path" you'd have "import stdlib.os.path". This is more or less Java's method and it's a rather good one. Unfortunately it's also a huge incompatible change so it's not going to happen.
• Introduce syntax to distinguish between imports of modules from the current directory (or below) and packages installed in the system. This is more or less C's method (the difference between #include <foo.h> and #include "foo.h"). However, that leads to nasty problems in the module cache (not a problem in C as modules don't get "cached" like that - although given C's complete lack of namespaces plenty of other problems can arise without careful naming!).
• Prevent imports from the current directory entirely (i.e. remove '' from sys.path). In other words, you're allowed a top level script (imported as __main__ as usual) but anything you want to import has to be installed as a package. Unfortunately this imposes a nasty burden on anyone wanting to break up a long script into manageable chunks and would probably just lead to people hacking '' back into sys.path anyway.

There's probably more but I can't think of them off the top of my head. Anyway, suffice it to say it's not a trivial problem that one can solve just by shuffling things around in sys.path. At this stage in Python's life (i.e. thoroughly mature and widely deployed) there's no good solution.

All I can say in its defence is that a pretty serious amount of thought has gone into the evolution of the import system over time: PEP-328, PEP-366, and PEP-395 to name but a few. They're worth a read to get an idea of how subtle and difficult language design is.


Dave.

[jojopi]

You can see the first entry there is '' which simply means the current path.

But you were running Python interactively. I stress again that within a script, sys.path will not normally contain the empty element, and import will not search the current directory, rather the directory where the script lives.

[waveform80]

You can see the first entry there is '' which simply means the current path.

But you were running Python interactively. I stress again that within a script, sys.path will not normally contain the empty element, and import will not search the current directory, rather the directory where the script lives.

Sorry, you're quite right - the first element will be the directory containing the script executed (e.g. '/home/dave') rather than '' indicating the current directory. However, where you're running a script from the current directory, the effect is exactly the same - Python will still find your numbers rather than the system's numbers (assuming no imported module changes the working directory which would be rather bad practice!).

Dave.

[butchec]

It's because Python's import hierarchy starts with files in the current folder before looking in the installed modules.

It is odd, though. I wonder if numpy should be using relative imports "import .number" rather than "import number"?

Indeed - the default Python search path looks something like this (from my Ubuntu machine):

Code: Select all

dave@spider:~/picraft$ python
Python 2.7.6 (default, Mar 22 2014, 22:59:56) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/dave/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client', '/usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode']

You can see the first entry there is '' which simply means the current path. If you look through the stack trace from the top you can get an idea of what's going on:

• You run something like "python numbers.py" so the Python interpreter starts up and places your module as "__main__" in sys.modules (which is the cache of loaded modules)
• numbers.py executes "from astro_pi import AstroPi". Python first checks whether "astro_pi" is in the sys.modules cache; it isn't so Python searches for the astro_pi package in sys.path and when it finds it, places "astro_pi" in sys.modules
• astro_pi/__init__.py executes "from .astro_pi import AstroPi" which means it wants to import the astro_pi module from within its package (also called astro_pi). Python searches for "astro_pi.astro_pi" in sys.modules, doesn't find it, and so searches for it under the astro_pi package's path (the "." makes this a relative import so it doesn't search the whole of sys.path). It's found and bunged in sys.modules as "astro_pi.astro_pi"
• astro_pi/astro_pi.py executes "import numpy as np". Python searches for "numpy" in sys.modules, doesn't find it, and searches sys.path for it. It's found and added to sys.modules as "numpy"
• numpy/__init__.py executes "from . import add_newdocs". This is another relative import so Python searches sys.modules for "numpy.add_newdocs", doesn't find it, then searches numpy's path for add_newdocs.py, finds it and adds it to sys.modules as "numpy.add_newdocs"
• Dave's fingers get tired from all this typing and I'm sure you've got the idea by now so things continue like this until...
• numpy/core/numerictypes.py attempts to execute "import numbers". Python searches sys.modules for "numbers", doesn't find it, searches sys.path and finds it in the current directory. Python adds it to sys.modules as "numbers". Numpy gets a module it didn't expect containing stuff it doesn't want and things go "bang"!

Now, Ben pointed out numpy ought to be doing a relative import of numbers, like "import .numbers" or "from . import numbers" to ensure that only numpy's path is searched and not the whole sys.path. That would indeed be sensible ... if the numbers module that numpy is looking for was part of numpy itself. Unfortunately it's not. Instead it's looking for the numbers module from the standard library; it's in Python 2 and 3 and it defines the abstract base types for numerals (integral, rational, real, etc.). Because numpy's looking for something outside itself it has no option but to use an absolute import, and thus the problem arises.

So, the solution is simply: don't name your scripts after top level packages in the standard library (and preferably top level third party packages too). Admittedly the standard library of Python is unusually large (the "batteries included" philosophy and all that) so this is harder than in other languages, but sticking with patterns like "my_numbers.py" or "numbers_1.py" tends to remove such problems (naming packages is a tougher problem!).

Now, onto Dougie's criticism and solution: just shove the current directory to the end of the sys.path and all will be well! In the scenario presented above that's absolutely correct. Unfortunately from the point of view of language design, it's just shoved the problem elsewhere...

Consider the case where the user now decides that "numbers" should be a module that their main script calls, so they've got a "main.py" and "numbers.py". They run "python main.py" and the first thing it does is "import numbers". Now Python searches sys.path and finds ... the standard library's numbers module instead of the user's expected module and things go "bang!" again. The core problem is still the same: the user has named a module the same thing as a top-level module/package - the only difference is when things go bang.

There are a number of "proper" solutions to this but for one reason or another none of them are going to happen:

• Ensure everything sits under a namespace unlikely to be duplicated. For example, stick the entire standard library in a "stdlib" package so instead of "import numbers" you'd have to do "import stdlib.numbers" and instead of "import os.path" you'd have "import stdlib.os.path". This is more or less Java's method and it's a rather good one. Unfortunately it's also a huge incompatible change so it's not going to happen.
• Introduce syntax to distinguish between imports of modules from the current directory (or below) and packages installed in the system. This is more or less C's method (the difference between #include <foo.h> and #include "foo.h"). However, that leads to nasty problems in the module cache (not a problem in C as modules don't get "cached" like that - although given C's complete lack of namespaces plenty of other problems can arise without careful naming!).
• Prevent imports from the current directory entirely (i.e. remove '' from sys.path). In other words, you're allowed a top level script (imported as __main__ as usual) but anything you want to import has to be installed as a package. Unfortunately this imposes a nasty burden on anyone wanting to break up a long script into manageable chunks and would probably just lead to people hacking '' back into sys.path anyway.

There's probably more but I can't think of them off the top of my head. Anyway, suffice it to say it's not a trivial problem that one can solve just by shuffling things around in sys.path. At this stage in Python's life (i.e. thoroughly mature and widely deployed) there's no good solution.

All I can say in its defence is that a pretty serious amount of thought has gone into the evolution of the import system over time: PEP-328, PEP-366, and PEP-395 to name but a few. They're worth a read to get an idea of how subtle and difficult language design is.


Dave.

Hi All,

Thank you for such a complete answer to the problem; it makes for some interesting reading...

I am going to make sure that I no longer name anything that even remotely has 'number' or 'numbers' in the filename.

Is there information anywhere that provides a list of the modules, e.g number etc..? This will help with any new programs that I create for the AstroPi.

Chris.

[waveform80]

...
Is there information anywhere that provides a list of the modules, e.g number etc..? This will help with any new programs that I create for the AstroPi.

Chris.

The list for the Python standard library can be found in the Python 2 docs and the Python 3 docs (yes, it's quite a long list - worth perusing though as there's some great built-in stuff!).

As for top-level packages that you've installed on your system (like numpy) ... that's a little more tricky (especially once you get into using virtual envs). The trivial way is to fire up "ipython" and use its tab-completion to find out by typing "import " and then hitting the Tab key (not exactly precise, that'll list modules in the working directory too). The more precise method would be to use the site module to enumerate them.

Dave.

[DougieLawson]

Is there information anywhere that provides a list of the modules, e.g number etc..? This will help with any new programs that I create for the AstroPi.

Your easiest way to check for name clashes is to use an interactive python shell. If you attempt to import foo then it will either barf with "No module named foo" or just work if foo is found in the sys.path.

Code: Select all

pi@aplus ~ $ python3
Python 3.2.3 (default, Mar  1 2013, 11:53:50)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import foo
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named foo
>>> import numbers
>>>

[ben_nuttall]

Thanks for weighing in Dave!