What are Python Subprocesses & Why Do They Matter?

Python has a lot of tasks and activities that happen in the background; these tasks are called processes. And, just like a species of snake can also have subspecies, the Python process can have subprocesses. While this information can feel daunting, the truth is that it has the power to make our work more manageable.

What is a subprocess in Python?

A subprocess is a task that is created by a process to complete a task within the parent process. Let’s expand on that; a process is a task performed by the program during its execution. In many, if not all cases, processes usually have smaller tasks that need to be completed before the process can end.

This is where subprocesses come into play. When a process has a minor task that needs to be completed, it can spawn a subprocess to handle that task while the process continues to run. This creates a more fluid and efficient way to manage data and memory.

If a new process were created for every task that a process needs to complete, devices would quickly run out of memory for the tasks to be completed. Furthermore, this methodology would be highly inefficient due to the workload it would ultimately put on device processors.

By creating small subprocesses to handle more straightforward tasks within a process, you will be able to run more processes simultaneously. This is called multiprocessing, a package added to the Python language to support the use of multiple processors. As an older language, Python did not help that type of performance since devices did not have as many processors back then.

History fact: The subprocess module was added in Python 2.4 to improve efficiency and replace outdated data and memory management approaches. Shortly after adding subprocesses, Python added a multiprocess module included in Python 2.6 and above. This addition of the multiprocess package aimed at further improving software performance on devices with many CPUs.

How to Use Subprocess in Python

Before using subprocesses in Python, let's talk about how it works with a relatively common example for computer users. All computers have a task manager; this program is designed for watching processes running on your computer. Your computer assigns some RAM and CPU space for each process running. This concept is common, but many users don’t know that these processes are often running tasks.

To use a subprocess in Python, you will need to create them.Python includes a method in the subprocess module called call, which starts a new process. The method accepts a command that will tell the subprocess what action it needs to complete.

For example, to start the windows notepad as a subprocess for a parent process, you would pass in the string your/path/to/program/notepad.exe, including its path. This would tell the subprocess to locate the notepad.exe file using the path provided and subsequently open and run the file.

 
import subprocess

code = subprocess.call("notepad.exe")

if code == 0:
print("Success!")
else:
print("Error!")

Success!

Python Subprocess Example

Subprocess comes with more than just the call method; it includes the Popen() method. The Popen method does not wait for the subprocess to end before returning information. To do this with Popen, you must use the wait() method after Popen. Let's look at three examples of how to use a subprocess.

In this example, you will see the subprocess used to ping a website and check its operation status.

Using Popen Method

The Popen method does not wait to complete the process to return a value. This means that the information you get back will not be complete. Instead, you will get a response that the initial task was started but nothing about the completion.

 
program = "notepad.exe"
subprocess.Popen(program)

/*response*/
<subprocess.Popen object at 0x01EE0430>

The above code successfully starts the program, but you will have received no information about the rest of the process operation. To get information about the entire process function and learn if there were any errors during runtime, you need to use the wait method.

Using Popen with the Wait Method

 
program = "notepad.exe"
process = subprocess.Popen(program)
code = process.wait()
print(code)

/*result*/
0

The wait method waits to return a value until the subprocess has finished and, in this case, returns zero, which means there were no errors.

There are cases in which you cannot use the wait() method as it can cause a deadlock freezing your program until it is resolved, specifically when using stdout/stderr=PIPE commands. In these cases instead of using wait(), you can use the communicate() method. Let’s look at how that is done next.

 
args = ["ping", "www.google.com"]
process = subprocess.Popen(args, stdout=subprocess.PIPE)
data = process.communicate()
print(data)

For the sake of this post, the result of this action is inconsequential to its operation. But it is worth noting the information returned by the communicate() method that this is a two-element tuple that contains what was in stdout and stderr.

Using the Subprocess Module in Your Software

By now, you should feel confident in understanding how Python subprocesses work and what they mean for your software development needs. There is still a lot to consider moving forward with subprocesses including other ways to interact with them. It is a vast subject, but you are more than ready to slither your way into learning even more about them. Check out these final thoughts on subprocesses to keep in mind as you push forward.

• Subprocesses provide an excellent way to maximize the efficiency of your code.
• Subprocesses can be invoked in two main ways, using the call() or Popen() methods.
• The wait() method is needed in order to capture the complete runtime result of a process when using the Popen() method.
• The wait() method can cause a deadlock when used with stdout/stderr=PIPE commands
• You can use the communicate() method when the wait method is not a good choice.

With these facts in hand and the above examples, you are ready to start using the subprocess module in your software.