F401 erroneously raised with "import as"

Bug #1589186 reported by mforbes
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Pyflakes
Incomplete
Undecided
Unassigned

Bug Description

The following code raises an F401 error.

```
import scipy.optimize
import scipy as sp

print(sp.optimize)
```

I don't see any obvious workaround. I used to do:

```
import scipy.optimize
sp = scipy

import numpy

print(sp.optimize)
```

but now flake8 correctly flags the line "sp = scipy" as E402 module level import not at top of file, so this workaround is less useful. (One could put all the module assignments at the end, but this is not very good for reading the code.)

I seem to recall discussing this at some point but cannot find the old discussion.

Revision history for this message
John Vandenberg (jayvdb) wrote :

So the flake8 error F401 is pyflakes reporting the following for line 1: 'scipy.optimize' imported but unused.

Which is correct for the sample code you have given.

scipy.optimize is unused, as the object with name 'scipy' is not accessed on line 2; instead the import on line 2 creates a new object 'sp' which just happens to be a module called 'scipy' in the import lookup system.

So I dont believe this is a bug, based on the sample code provided.

I am guessing that you dont really want to print scipy.optimize ? You are trying to hide the error, by doing a dummy usage?

The following will do the import, and hide the pyflakes error

import scipy.optimize as _
import scipy as sp

del _

def x():
    return sp.foo()

If you really need to print or access scipy.optimize, the this works

import scipy.optimize
import scipy as sp

print(scipy.optimize)

def x():
    return sp.foo()

But maybe your real code cant be solved those ways, in which case let me know the project and I'll take a look at the problem with real code.

Changed in pyflakes:
status: New → Incomplete
Revision history for this message
mforbes (mforbes-physics) wrote :

The print line is simply a way to "use" scipy.optimize. A "real" usage (MWE) would be

```
import scipy.optimize
import scipy as sp

res = sp.optimize.root(lambda x: x, 1)
```

This has the same problem. Your example with del _ is a indeed a workaround (though it is pretty ugly!)

It is extremely common to use scipy, numpy, and matplotlib via:

import numpy as np
import scipy as sp
from matplotlib import pyplot as plt

then to use np. sp. or plt. in ones code. However, especially with scipy, the top level import does not bring in the whole library, hence the need for

import scipy.optimize

Although this might be difficult to detect, I believe it is a real bug since scipy.optimize is indeed used (just through the standard alias sp.optimize)

Revision history for this message
John Vandenberg (jayvdb) wrote :

Then a cleaner workaround would be something like

import scipy.optimize
import scipy as sp

assert sp.optimize == scipy.optimize

--

or even just `assert scipy` will do the trick, and will work for multiple scipy.x imports.

The difficulty is that from a symbol/name perspective, which is what pyflakes mostly uses, the name `scipy` is created in the module and never used.

But I am seeing your point .. we know that `import scipy.optimize` adds `optimize` to what will be later named `sp`, and it is used with that alias.

The related problem is that pyflakes doesnt yet do any sanity checking on submodule imports (import x.y). i.e. the following passes

import scipy.foo
import scipy.bar
scipy.baz

It should have errors about `scipy.foo` and `scipy.bar` being unused, and `scipy` on line 3 being an implicit import of `scipy`.

But maybe we can ignore the fact that submodule import usage tracking is not working, and focus on avoiding the error in your scenario of using `import x as y`. i.e. this passes

import scipy.optimize
import scipy
assert scipy

In the above, the 'optimize' part is ignored by pyflakes, so why shouldnt the following also pass:

import scipy.optimize
import scipy as sp
assert sp

I think we have enough information to allow the above the pass, without any regressions.

Revision history for this message
mforbes (mforbes-physics) wrote :

Again, the reason I am now having an issue is that flake8 adds "E402 module level import not at top of file" which requires the assert statement to come after all the imports breaking locality.

It would be really nice if pyflakes could track the rename of scipy to py, but I think that the best workaround at this point is to del all the unused modules at the end of the import section:

```
import scipy.optimize
import scipy as sp

import numpy.linalg
import numpy as np

...

del scipy, numpy

...
```

Then the statement at least has some meaning - use sp., not scipy. etc. in the code, following conventions.

Revision history for this message
asmeurer (asmeurer) wrote :

The problem is that pyflakes treats imports like variable assignments, but they aren't quite like that. If you write

a = func()

and then never use a, pyflakes is right to tell you that a is never used, because you could just as well have written

func()

without any assignment. But "import scipy.optimize" is the only way to load the scipy.optimize module (it isn't loaded with "import scipy" for performance purposes). So the purpose of the line is to load a module, but it also happens to load a name into the namespace.

Secondly, it's best practice to import scipy and numpy as sp and np, respectively.

> and `scipy` on line 3 being an implicit import of `scipy`.

I don't think that's an error. Python explicitly runs the top-level __init__.py and puts the module name in the namespace. For instance, this code works just fine

import numpy.linalg
numpy.array

Revision history for this message
John Vandenberg (jayvdb) wrote :

I agree pyflakes can special case submodule imports, as they cant be done another way.
Unless anyone objects, I will do the fix, which I suspect is a one-liner now.

> > and `scipy` on line 3 being an implicit import of `scipy`.
>
> I don't think that's an error. Python explicitly runs the top-level __init__.py and puts the module name in the namespace. For instance, this code works just fine
>
> import numpy.linalg
> numpy.array

While this is possible, it can also be a problem in complex code. Pywikibot had one such gnarly problem (https://phabricator.wikimedia.org/T113161) which I should extract out into a simple example of how implicit imports can be problematic.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.