Multi Threading
An iPhone has a CPU, its Central Processing Unit. Technically, a CPU can only perform one operation at a time – once per clock cycle. Multithreading allows the processor to create concurrent threads it can switch between, so multiple tasks can be executed at the same time.
Multithreading and Concurrency
An iPhone has a CPU, its Central Processing Unit. Technically, a CPU can only perform one operation at a time – once per clock cycle. Multithreading allows the processor to create concurrent threads it can switch between, so multiple tasks can be executed at the same time.
It appears as if the two threads are executed at the same time, because the processor switches rapidly between executing them. As a smartphone or desktop user, you don't notice the switches because they occur so rapidly. Performing multiple tasks at the same time is called concurrency.
Concurrency is the notion of multiple things happening at the same time. This is generally achieved either via time-slicing, or truly in parallel if multiple CPU cores are available to the host operating system.
A common example is the UI thread. Ever noticed how apps can still appear responsive and have smooth animations, even though the CPU is doing a lot of work? That's because of multithreading.
On an iPhone, the UI always has its own thread. When you're downloading a file from the web, or executing a complex task, the UI remains responsive and smooth.
This is because the iPhone switches between drawing the screen and downloading the file quickly, until both tasks are complete. There's enough parallel CPU power available to draw the screen and download the file, without having to execute the tasks sequentially.
Then what about multicore CPUs? Such a CPU, often called a System-on-a-Chip (SoC), typically has multiple CPUs in one. This allows for truly parallel processing, instead of the concurrency by rapidly switching of multithreading.
In reality, multicore processing isn't exactly parallel. It depends greatly on the type of work the processors are executing – but that's a topic for another article.
To summarize, multithreading allows a CPU to rapidly switch between multiple tasks in such a way that it appears as if the tasks are executed simultaneously.
Don't Block The UI Thread!
You can't update an app's UI outside the main thread. User Interface operations, like showing a dialog or updating the text on a button can only be performed on the main thread. But… why?
There are a number of reasons for that, but the foremost reason is to avoid race conditions. A race condition occurs when two tasks are executed concurrently, when they should be executed sequentially in order to be done correctly. Conversely, some tasks should only be dispatched asynchronously to avoid a deadlock.
Drawing the UI on the screen of the iPhone is such a sequential task. The same goes for resolving Auto Layout constraints. You can't change the constraints while they're being calculated.
What if you execute a synchronous, blocking task on the main UI thread? The thread will wait until the synchronous task is executed. As a result, your apps UI may stutter, lag or become unresponsive for some time.
We can derive a few first principles from this:
Executing Async Code with Grand Central Dispatch
When you need tasks executed concurrently, should you just go about creating a bunch of threads? Fortunately not! iOS has a terrific mechanism for working with concurrent tasks called Grand Central Dispatch.
The name "Grand Central Dispatch" is a reference to Grand Central Terminal in downtown New York. Imagine the CPU as a bunch of railroads, with the traincars as tasks being executed on those lines. A dispatcher is moving the cars along the railroads to ensure that they reach their destination quickly on a limited number of lines.
Grand Central Dispatch is a wrapper around low-level code, to create threads and manage code. Its emphasis is on dispatching, i.e. making sure that a number of tasks of variable importance and length are executed in a timeframe as reasonable as possible.
It appears as if the two threads are executed at the same time, because the processor switches rapidly between executing them. As a smartphone or desktop user, you don't notice the switches because they occur so rapidly. Performing multiple tasks at the same time is called concurrency.
Concurrency is the notion of multiple things happening at the same time. This is generally achieved either via time-slicing, or truly in parallel if multiple CPU cores are available to the host operating system.
A common example is the UI thread. Ever noticed how apps can still appear responsive and have smooth animations, even though the CPU is doing a lot of work? That's because of multithreading.
On an iPhone, the UI always has its own thread. When you're downloading a file from the web, or executing a complex task, the UI remains responsive and smooth.
This is because the iPhone switches between drawing the screen and downloading the file quickly, until both tasks are complete. There's enough parallel CPU power available to draw the screen and download the file, without having to execute the tasks sequentially.
Then what about multicore CPUs? Such a CPU, often called a System-on-a-Chip (SoC), typically has multiple CPUs in one. This allows for truly parallel processing, instead of the concurrency by rapidly switching of multithreading.
In reality, multicore processing isn't exactly parallel. It depends greatly on the type of work the processors are executing – but that's a topic for another article.
To summarize, multithreading allows a CPU to rapidly switch between multiple tasks in such a way that it appears as if the tasks are executed simultaneously.
Don't Block The UI Thread!
You can't update an app's UI outside the main thread. User Interface operations, like showing a dialog or updating the text on a button can only be performed on the main thread. But… why?
There are a number of reasons for that, but the foremost reason is to avoid race conditions. A race condition occurs when two tasks are executed concurrently, when they should be executed sequentially in order to be done correctly. Conversely, some tasks should only be dispatched asynchronously to avoid a deadlock.
Drawing the UI on the screen of the iPhone is such a sequential task. The same goes for resolving Auto Layout constraints. You can't change the constraints while they're being calculated.
What if you execute a synchronous, blocking task on the main UI thread? The thread will wait until the synchronous task is executed. As a result, your apps UI may stutter, lag or become unresponsive for some time.
We can derive a few first principles from this:
- Keep the UI thread free, to keep your app responsive
- Avoid blocking the UI thread with synchronous code
Executing Async Code with Grand Central Dispatch
When you need tasks executed concurrently, should you just go about creating a bunch of threads? Fortunately not! iOS has a terrific mechanism for working with concurrent tasks called Grand Central Dispatch.
The name "Grand Central Dispatch" is a reference to Grand Central Terminal in downtown New York. Imagine the CPU as a bunch of railroads, with the traincars as tasks being executed on those lines. A dispatcher is moving the cars along the railroads to ensure that they reach their destination quickly on a limited number of lines.
Grand Central Dispatch is a wrapper around low-level code, to create threads and manage code. Its emphasis is on dispatching, i.e. making sure that a number of tasks of variable importance and length are executed in a timeframe as reasonable as possible.
DispatchQueue.global(qos: .userInitiated).async {
// Download file or perform expensive task asynchronously
DispatchQueue.main.async {
// Update the UI
}
}
if a software that doesn't take advantage of all the computational power at its disposal is going to create these freezes whenever it needs to do something resource-intensive.
Anything related to file I/O, data processing, or networking usually warrants a background task
(unless you have a very compelling excuse to halt the entire program). There aren't many reasons that these tasks should block your user from interacting with the rest of your application. Consider how much better the user experience of your app could be if instead, the profiler reported something like this:
DispatchQueue.global(qos: .userInitiated).async {
// Download file or perform expensive task asynchronously
DispatchQueue.main.async {
// Update the UI
}
}
Analyzing an image, processing a document or a piece of audio, or writing a sizeable chunk of data to disk are examples of tasks that could benefit greatly from being delegated to background threads.
Quality-of-Service (QoS) and Grand Central Dispatch
In essence, the QoS tells the dispatcher how important the task that's dispatched is. More important tasks are executed earlier on the dispatch queue, have more system resources available, and thus complete quicker than the same task with a lower priority.
Higher QoS tasks use more battery power, so defining the right QoS ensures that your app uses power and memory as efficiently as possible. Avoid the urge to mark every task as "important," because it'll only mark them as equally important and thus negate the entire priority system.
It's important that Quality-of-Service and priority aren't exactly the same. iOS will give tasks with a higher QoS more priority, but it'll also throttle tasks up and down based on other aspects of the OS environment.
iOS has four Quality-of-Service levels, from highest to lowest priority:
In essence, the QoS tells the dispatcher how important the task that's dispatched is. More important tasks are executed earlier on the dispatch queue, have more system resources available, and thus complete quicker than the same task with a lower priority.
Higher QoS tasks use more battery power, so defining the right QoS ensures that your app uses power and memory as efficiently as possible. Avoid the urge to mark every task as "important," because it'll only mark them as equally important and thus negate the entire priority system.
It's important that Quality-of-Service and priority aren't exactly the same. iOS will give tasks with a higher QoS more priority, but it'll also throttle tasks up and down based on other aspects of the OS environment.
iOS has four Quality-of-Service levels, from highest to lowest priority:
- .userInteractive is intended for user-interactive tasks, such as animations, events, and updating UI. As a rule of thumb, use this QoS for tasks that your app's user is actively using.
- .userInitiated is intended for tasks that prevent the user from using your app, like saving a file. Use it for tasks that your app's user is actively waiting on.
- .utility is intended for tasks that don't require an immediate result, such as background downloads with a progress bar. Use it for background tasks that need a balance between responsiveness, performance and energy efficiency.
- .background is intended for tasks that aren't visible to your app's user, like indexing, sync and backups. This setting prioritizes energy efficiency.
Queues and Grand Central Dispatch
A dispatch queue is simply a pool of tasks. Grand Central Dispatch will determine which tasks are executed. It'll also determine how much CPU time those tasks get. More CPU time means a quicker completion.
Think back to that train terminal dispatcher. Which traincar is more important; which should get to its destination the quickest? The dispatcher can only move one car at a time, and moving one car forward more means delaying all the other cars.
The dispatcher takes an informed decision and lets all tasks run until they complete, giving preference to more important tasks (QoS). It'll also assign tasks to different CPUs – the iPhone 11 Pro, for example, has a multicore SoC with 6 CPUs. Execute two tasks on two CPU cores, and they're executed in parallel!
Finally, in the example above, another task is dispatched to the main thread. You need to inform the user, for instance, that their download has completed. This is something you need to do on the main thread.
It's compelling to think that tasks on a dispatch queue are executed one after the other, and you wouldn't be wrong. After all, when we queue up at a supermarket or icecream parlor each of us gets served after the other. However, a queue has a prioritization mechanism based on Quality-of-Service. A background task is less important than a user-interactive task, so the user-interactive task gets more CPU power and completes sooner.
Fortunately you don't have to manage all those priorities, queues, threads and tasks yourself. That's the kind of managing Grand Central Dispatch does!
A dispatch queue is simply a pool of tasks. Grand Central Dispatch will determine which tasks are executed. It'll also determine how much CPU time those tasks get. More CPU time means a quicker completion.
Think back to that train terminal dispatcher. Which traincar is more important; which should get to its destination the quickest? The dispatcher can only move one car at a time, and moving one car forward more means delaying all the other cars.
The dispatcher takes an informed decision and lets all tasks run until they complete, giving preference to more important tasks (QoS). It'll also assign tasks to different CPUs – the iPhone 11 Pro, for example, has a multicore SoC with 6 CPUs. Execute two tasks on two CPU cores, and they're executed in parallel!
Finally, in the example above, another task is dispatched to the main thread. You need to inform the user, for instance, that their download has completed. This is something you need to do on the main thread.
It's compelling to think that tasks on a dispatch queue are executed one after the other, and you wouldn't be wrong. After all, when we queue up at a supermarket or icecream parlor each of us gets served after the other. However, a queue has a prioritization mechanism based on Quality-of-Service. A background task is less important than a user-interactive task, so the user-interactive task gets more CPU power and completes sooner.
Fortunately you don't have to manage all those priorities, queues, threads and tasks yourself. That's the kind of managing Grand Central Dispatch does!
Concurrency vs Parallelism
In the first situation presented above, we observe that tasks can run concurrently, but not in parallel. This is similar to having multiple conversations in a chatroom, and interleaving (context-switching) between them, but never truly conversing with two people at the same time. This is what we call concurrency. It is the illusion of multiple things happening at the same time when in reality, they're switching very quickly. Concurrency is about dealing with lots of things at the same time. Contrast this with the parallelism model, in which both tasks run simultaneously. Both execution models exhibit multithreading, which is the involvement of multiple threads working towards one common goal. Multithreading is a generalized technique for introducing a combination of concurrency and parallelism into your program.
In the first situation presented above, we observe that tasks can run concurrently, but not in parallel. This is similar to having multiple conversations in a chatroom, and interleaving (context-switching) between them, but never truly conversing with two people at the same time. This is what we call concurrency. It is the illusion of multiple things happening at the same time when in reality, they're switching very quickly. Concurrency is about dealing with lots of things at the same time. Contrast this with the parallelism model, in which both tasks run simultaneously. Both execution models exhibit multithreading, which is the involvement of multiple threads working towards one common goal. Multithreading is a generalized technique for introducing a combination of concurrency and parallelism into your program.
Concurrency vs Parallelism
In the first situation presented above, we observe that tasks can run concurrently, but not in parallel. This is similar to having multiple conversations in a chatroom, and interleaving (context-switching) between them, but never truly conversing with two people at the same time. This is what we call concurrency. It is the illusion of multiple things happening at the same time when in reality, they're switching very quickly. Concurrency is about dealing with lots of things at the same time. Contrast this with the parallelism model, in which both tasks run simultaneously. Both execution models exhibit multithreading, which is the involvement of multiple threads working towards one common goal. Multithreading is a generalized technique for introducing a combination of concurrency and parallelism into your program.
Grand Central Dispatch
iOS takes an asynchronous approach to solving the concurrency problem of managing threads. Asynchronous functions are common in most programming environments, and are often used to initiate tasks that might take a long time, like reading a file from the disk, or downloading a file from the web. When invoked, an asynchronous function executes some work behind the scenes to start a background task, but returns immediately, regardless of how long the original task might takes to actually complete.
A core technology that iOS provides for starting tasks asynchronously is Grand Central Dispatch (or GCD for short). GCD abstracts away thread management code and moves it down to the system level, exposing a light API to define tasks and execute them on an appropriate dispatch queue. GCD takes care of all thread management and scheduling, providing a holistic approach to task management and execution, while also providing better efficiency than traditional threads.
Serial Queues: The Main Thread
If we run a heavy task on the main thread , we will completely block the main thread, and therefore the entire interface
A core technology that iOS provides for starting tasks asynchronously is Grand Central Dispatch (or GCD for short). GCD abstracts away thread management code and moves it down to the system level, exposing a light API to define tasks and execute them on an appropriate dispatch queue. GCD takes care of all thread management and scheduling, providing a holistic approach to task management and execution, while also providing better efficiency than traditional threads.
- DispatchQueue.main: The main thread, or the UI thread, is backed by a single serial queue. All tasks are executed in succession, so it is guaranteed that the order of execution is preserved. It is crucial that you ensure all UI updates are designated to this queue, and that you never run any blocking tasks on it. We want to ensure that the app's run loop (called CFRunLoop) is never blocked in order to maintain the highest framerate. Subsequently, the main queue has the highest priority, and any tasks pushed onto this queue will get executed immediately.
- DispatchQueue.global: A set of global concurrent queues, each of which manage their own pool of threads. Depending on the priority of your task, you can specify which specific queue to execute your task on, although you should resort to using default most of the time. Because tasks on these queues are executed concurrently, it doesn't guarantee preservation of the order in which tasks were queued
Serial Queues: The Main Thread
If we run a heavy task on the main thread , we will completely block the main thread, and therefore the entire interface
private func compute() -> Void {
var counter = 0
for _ in 0..<9999999 {
counter += 1
}
}
We can see that the Main Thread is clearly at 100% capacity for almost 5 seconds.
Background Threads
How can we make this better? DispatchQueue.global() to the rescue! This is where background threads come in. Referring to the GCD architecture diagram above, we can see that anything that is not the Main Thread is a background thread in iOS. They can run alongside the Main Thread, leaving it fully unoccupied and ready to handle other UI events like scrolling, responding to user events, animating etc.
Keep in mind that we aren't really managing threads here. We're submitting tasks (in the form of closures or blocks) to the desired queue with the assumption that it is guaranteed to execute at some point in time. The queue decides which thread to allocate the task to, and it does all the hard work of assessing system requirements and managing the actual threads. This is the magic of Grand Central Dispatch.
Wrap the code execution with DispatchQueue.global:
Keep in mind that we aren't really managing threads here. We're submitting tasks (in the form of closures or blocks) to the desired queue with the assumption that it is guaranteed to execute at some point in time. The queue decides which thread to allocate the task to, and it does all the hard work of assessing system requirements and managing the actual threads. This is the magic of Grand Central Dispatch.
Wrap the code execution with DispatchQueue.global:
DispatchQueue.global(qos: .userInitiated).async { [unowned self] in
self.compute()
}
Global thread
Looking at the profiler again, it's quite clear to us that this is a huge improvement. The task takes an identical amount of time, but this time, it's happening in the background without locking up the UI. Even though our app is doing the same amount of work, the perceived performance is much better because the user will be free to do other things while the app is processing.
You may have noticed that we accessed a global queue of .userInitiated priority. This is an attribute we can use to give our tasks a sense of urgency. If we run the same task on a global queue of and pass it a qos attribute of background , iOS will think it's a utility task, and thus allocate fewer resources to execute it. So, while we don't have control over when our tasks get executed, we do have control over their priority.
The main dispatch queue is a globally available serial queue that executes tasks on the application's main thread. Because it runs on your application's main thread, the main queue is often used as a key synchronization point for an application.
You may have noticed that we accessed a global queue of .userInitiated priority. This is an attribute we can use to give our tasks a sense of urgency. If we run the same task on a global queue of and pass it a qos attribute of background , iOS will think it's a utility task, and thus allocate fewer resources to execute it. So, while we don't have control over when our tasks get executed, we do have control over their priority.
The main dispatch queue is a globally available serial queue that executes tasks on the application's main thread. Because it runs on your application's main thread, the main queue is often used as a key synchronization point for an application.
Concurrency vs Parallelism
Concurrent Threads
So far, our tasks have been executed exclusively in a serial manner. DispatchQueue.main is by default a serial queue, and DispatchQueue.global gives you four concurrent dispatch queues depending on the priority parameter you pass in.
Let's say we want to take five images, and have our app process them all in parallel on background threads. How would we go about doing that? We can spin up a custom concurrent queue with an identifier of our choosing, and allocate those tasks there. All that's required is the .concurrent attribute during the construction of the queue.
Let's say we want to take five images, and have our app process them all in parallel on background threads. How would we go about doing that? We can spin up a custom concurrent queue with an identifier of our choosing, and allocate those tasks there. All that's required is the .concurrent attribute during the construction of the queue.
class ViewController: UIViewController {
let queue = DispatchQueue(label: "com.app.concurrentQueue", attributes: .concurrent)
let images: [UIImage] = [UIImage].init(repeating: UIImage(), count: 5)
@IBAction func handleTap(_ sender: Any) {
for img in images {
queue.async { [unowned self] in
self.compute(img)
}
}
}
private func compute(_ img: UIImage) -> Void {
// Pretending to post-process a large image.
var counter = 0
for _ in 0..<9999999 {
counter += 1
}
}
}Parallelization of N Tasks
Semaphores are signaling mechanisms. They are commonly used to control access to a shared resource. Imagine a scenario where a thread can lock access to a certain section of the code while it executes it, and unlocks after it's done to let other threads execute the said section of the code. You would see this type of behavior in database writes and reads, for example. What if you want only one thread writing to a database and preventing any reads during that time? This is a common concern in thread-safety called Readers-writer lock. Semaphores can be used to control concurrency in our app by allowing us to lock n number of threads.
let semaphore = DispatchSemaphore(value: 1)
DispatchQueue.global().async {
print("Wait")
semaphore.wait()
print("Wait finished")
sleep(1) // Executing
semaphore.signal()
print("Done")
}GCD is great when you want to dispatch one-off tasks or closures into a queue in a 'set-it-and-forget-it' fashion, and it provides a very lightweight way of doing so. But what if we want to create a repeatable, structured, long-running task that produces associated state or data? And what if we want to model this chain of operations such that they can be cancelled, suspended and tracked, while still working with a closure-friendly API?
- You may want to create dependencies between tasks, and while you could do this via GCD, you're better off defining them concretely as Operation objects, or units of work, and pushing them onto your own queue. This would allow for maximum reusability since you may use the same pattern elsewhere in an application.
- The Operation and OperationQueue classes have a number of properties that can be observed, using KVO (Key Value Observing). This is another important benefit if you want to monitor the state of an operation or operation queue.
- Operations can be paused, resumed, and cancelled. Once you dispatch a task using Grand Central Dispatch, you no longer have control or insight into the execution of that task. The Operation API is more flexible in that respect, giving the developer control over the operation's life cycle.
- OperationQueue allows you to specify the maximum number of queued operations that can run simultaneously, giving you a finer degree of control over the concurrency aspects.
So why not opt for a higher level abstraction and avoid using GCD entirely? While GCD is ideal for inline asynchronous processing, Operation provides a more comprehensive, object-oriented model of computation for encapsulating all of the data around structured, repeatable tasks in an application. Developers should use the highest level of abstraction possible for any given problem, and for scheduling consistent, repeated work, that abstraction is Operation. Other times, it makes more sense to sprinkle in some GCD for one-off tasks or closures that we want to fire. We can mix both OperationQueue and GCD to get the best of both worlds.
let blockOperation = BlockOperation {
print("Executing!")
}
let queue = OperationQueue()
queue.addOperation(blockOperation)
OR
queue.addOperation {
print("Executing!")
}
Concurrency vs Parallelism
The Cost of Concurrency
DispatchQueue and friends are meant to make it easier for the application developer to execute code concurrently. However, these technologies do not guarantee improvements to the efficiency or responsiveness in an application. It is up to you to use queues in a manner that is both effective and does not impose an undue burden on other resources. For example, it's totally viable to create 10,000 tasks and submit them to a queue, but doing so would allocate a nontrivial amount of memory and introduce a lot of overhead for the allocation and deallocation of operation blocks. This is the opposite of what you want! It's best to profile your app thoroughly to ensure that concurrency is enhancing your app's performance and not degrading it.
We've talked about how concurrency comes at a cost in terms of complexity and allocation of system resources, but introducing concurrency also brings a host of other risks like:
DispatchQueue and friends are meant to make it easier for the application developer to execute code concurrently. However, these technologies do not guarantee improvements to the efficiency or responsiveness in an application. It is up to you to use queues in a manner that is both effective and does not impose an undue burden on other resources. For example, it's totally viable to create 10,000 tasks and submit them to a queue, but doing so would allocate a nontrivial amount of memory and introduce a lot of overhead for the allocation and deallocation of operation blocks. This is the opposite of what you want! It's best to profile your app thoroughly to ensure that concurrency is enhancing your app's performance and not degrading it.
We've talked about how concurrency comes at a cost in terms of complexity and allocation of system resources, but introducing concurrency also brings a host of other risks like:
- Deadlock: A situation where a thread locks a critical portion of the code and can halt the application's run loop entirely. In the context of GCD, you should be very careful when using the dispatchQueue.sync { } calls as you could easily get yourself in situations where two synchronous operations can get stuck waiting for each other.
- Priority Inversion: A condition where a lower priority task blocks a high priority task from executing, which effectively inverts their priorities. GCD allows for different levels of priority on its background queues, so this is quite easily a possibility.
- Producer-Consumer Problem: A race condition where one thread is creating a data resource while another thread is accessing it. This is a synchronization problem, and can be solved using locks, semaphores, serial queues, or a barrier dispatch if you're using concurrent queues in GCD.
- ...and many other sorts of locking and data-race conditions that are hard to debug! Thread safety is of the utmost concern when dealing with concurrency.