NB: Scope

Programming for Data Science

What is Scope?

Scope is a fundamental concept in computer programming.

Scope is a property of a variable or other object that refers to where it can be seen in a program.

To be seen, or visible, means to be available or usable.

If a variable is in scope to a function, it is visible to the function.

If it is out of scope to a function, it is not visible to the function.

Scope hinges on the fact that programs have regions within which variables can be assigned.

These regions consist of nested code blocks.

We saw this distinction when looking at how function arguments work.

In the code below, x, y, and z are inside the function, while a, b, and c are outside of it.

def my_function(x, y):
    z = x + y
    return z

a = 10
b = 20
c = my_function(x=a, y=b)

The code inside the function block is local to the function.

The code outside the function block is global to the program.

To put it visuallly, the area contained by the red rectangle is local and the area of the green rectangle is global.

Not that indentation does the work of defining the regions by which scope is defined.

Lutz on Scoping

Here’s an excerpt from Lutz, Chapter 17. Please read the whole thing.

The enclosing module is a global scope. Each module is a global scope—that is, a namespace in which variables created (assigned) at the top level of the module file live. Global variables become attributes of a module object to the outside world after imports but can also be used as simple variables within the module file itself.

The global scope spans a single file only. Don’t be fooled by the word “global” here—names at the top level of a file are global to code within that single file only. There is really no notion of a single, all-encompassing global file-based scope in Python. Instead, names are partitioned into modules, and you must always import a module explicitly if you want to be able to use the names its file defines. When you hear “global” in Python, think “module.”

Assigned names are local unless declared global or nonlocal. By default, all the names assigned inside a function definition are put in the local scope (the namespace associated with the function call). If you need to assign a name that lives at the top level of the module enclosing the function, you can do so by declaring it in a global statement inside the function. If you need to assign a name that lives in an enclosing def, as of Python 3.X you can do so by declaring it in a nonlocal statement.

All other names are enclosing function locals, globals, or built-ins. Names not assigned a value in the function definition are assumed to be enclosing scope locals, defined in a physically surrounding def statement; globals that live in the enclosing module’s namespace; or built-ins in the predefined built-ins module Python provides.

Each call to a function creates a new local scope. Every time you call a function, you create a new local scope—that is, a namespace in which the names created inside that function will usually live. You can think of each def statement (and lambda expression) as defining a new local scope, but the local scope actually corresponds to a function call. Because Python allows functions to call themselves to loop—an advanced technique known as recursion and noted briefly in Chapter 9 when we explored comparisons—each active call receives its own copy of the function’s local variables. Recursion is useful in functions we write as well, to process structures whose shapes can’t be predicted ahead of time; we’ll explore it more fully in Chapter 19.

There are actually other scope levels.

Here is a nice visualization of them:

Please read Lutz, Chapter 17 for a good overview of scoping.

The Tinted Glass Metaphor

Code regions in a program are like vehicles with tinted glass:

Passengers can see outside, but outsiders can’t see inside.
Passengers in a vehicle can let outsiders look in by lowering the window.

Example 1

A variable x i defined outside a function but used inside of it.

In the code below x is global and seen from inside the function.

The variable a is local to the function.

Trying to print outside function throws error.

x = 10

def scope_func1(a):
    out = x + a
    return out

The following works because x is global and functions can access globals so long as they don’t reassign the variable name.

y = scope_func1(6)
print(y)

The following fails because a local and not visible outside the function.

print(a)

NameError: name 'a' is not defined

Example 2

A variable x defined both outside and inside function, and used inside the function.

This function reassigns x, so it becomes local.

A variable becomes local once it is used in an assignment statement within a function, or if it is the name of an argument.

x = 10 # Global

def scope_func2(a):
    x = 20 # Local
    print('x from inside:', x)
    out = x + a
    return out

y = scope_func2(6)
print(y)
print('x from outside:', x)

x from inside: 20
26
x from outside: 10

Example 2a

Here we define x as an argument.

This has the same effect as defining it in the body of the function.

x = 10 # Global

def scope_func2a(a, x=20): # Argment variables are local
    print('x from inside:', x)
    out = x + a
    return out

y = scope_func2a(6)
print(y)
print('x from outside:', x)

x from inside: 20
26
x from outside: 10

As an argument, though, it can be assigned the value of the global.

Nevertheless, only the value is being passed between the global and local versions of x.

x = 10 # Global

def scope_func2a(a, x=20):

    print('x from inside before incremenet:', x)
    x += 10
    print('x from inside after incremenet:', x)
    
    out = x + a
    
    return out

y = scope_func2a(6, x=x)
print(y)
print('x from outside :', x)

x from inside before incremenet: 10
x from inside after incremenet: 20
26
x from outside : 10

Example 3

A variable x defined both outside and inside function, and used inside the function in both global and local modes.

This one is interesting. It fails, but it’s not clear why at first.

x = 10

def scope_func3(a):
    print('x from fcn, before local definition:', x) # Global use of x
    x = 20 # Local use of x
    print('x from fcn, after local definition:', x)
    out = x + a
    return out

print('x from outside before local definition:', x)
scope_func3(6)
print('x from outside after local definition:', x)

x from outside before local definition: 30

UnboundLocalError: local variable 'x' referenced before assignment

The problem is that Python knows you are assigning x within the local scope of the function, so it won’t allow you to use it until it is defined.

The error can be fixed by referencing x as global inside function.

x = 10

def scope_func4(a):
    global x
    
    print('x from inside, before local definition:', x)
    x = 20
    print('x from inside, after local definition:', x)
    
    out = x + a
    return out

y = scope_func4(6)
print(y)
print('x from outside:', x)

x from inside, before local definition: 10
x from inside, after local definition: 20
26
x from outside: 20

Local / global conflicts

What will calling guess() do?

x = 10

def guess():
    x += 10 
    print(x)

guess()

UnboundLocalError: local variable 'x' referenced before assignment

Consider the following expression, which is the same as the unary operation inside of the function guess().

x = x + 10

The x on the left is local, since it is being defined inside the function.

However, the x on the right is assumed to already be defined, and so is global.

In effect, Python is presented with a contradiction and so throws an error.

We will see that R does not do this; it just goes with the global.

Nonlocal

If a variable is assigned in an enclosing def, it is nonlocal to nested functions.

The nonlocal keyword is similar to global, except that it refers to the scope of the enclosing function, not the script that contains the funtions.

x = 10 # Global
def func1(): # Enclosing function
    x = 20 # Local to function; "Nonlocal" to nested function
    def func1a():
        x = 30 # Local to nested function
        print(x)
    func1a()
    print(x)

print(x)
func1()
print(x)

x = 10
def func2():
    x = 20
    def func2a():
        nonlocal x
        x = 30
        print(x)
    func2a()
    print(x)

print(x) # 1
func2()
print(x) # 4

x = 10
def func3():
    x = 20
    def func3a():
        global x
        x = 30
        print(x)
    func3a()
    print(x)

print(x)
func3()
print(x)

Namespaces

You may wonder how Python keeps track to two variables with the same name in a program.

For example, when x is defined both globally and locally within the same program.

Python solves this problem by giving each region a name.

The region becomes a namespace.

With namespaces, a unique name to associated with each and every object in a Python program.

Namespacea work like the file system on a computer.

Just as you can have many files of the same name as long as they are in different folders.

The complete name of the file is actually the filename and the names of its parent folders, i.e. the path to the file in the file system.

`globals()` and `locals()`

Python keeps track of objects and namespaces in a dictionary. You can access local and global variables with globals() and locals().

def foo():
    x = 1
    print(locals())

foo()

{'x': 1}

G = globals()

G['foo']

<function __main__.foo()>

[key for key in globals().keys() if not key.startswith('_')]

['In',
 'Out',
 'get_ipython',
 'exit',
 'quit',
 'x',
 'scope_func2',
 'y',
 'scope_func2a',
 'scope_func1',
 'scope_func3',
 'scope_func4',
 'guess',
 'func1',
 'func2',
 'func3',
 'foo',
 'G']

Some Visualizations

The same object name can be present in multiple namespaces as isolation between the same name is maintained by their namespace.

Source: “Namespaces and Scope in Python” (GFG).