Apple has very strict guidelines for how—and when—apps can run in the background.
The following are the only scenarios in which a third-party iOS app can use CPU in the background.
Background Fetch
Any app can use "Background Fetch" (aka Background app refresh) to fetch content in the background for a few seconds, about once a day (on a timeframe regulated by iOS, out of the developer's control). Background Fetch takes up very little battery and CPU, and is mainly used by news/weather/social media/mail apps to update your feed so that when you open the app it won't be completely out of date.
Audio, Voice over IP, Bluetooth Communications
If an app plays audio in the background (over AirPlay or through the phone's speakers), iOS permits it to run in the background until it ceases to play the music; if an app allows you to make data-based phone calls (like Whatsapp or Skype calls) in the background, it can stay active, using CPU for the duration of the call; and if an app communicates to Bluetooth Accessories (like the Pebble app communicating with the Pebble Smartwatch), it can stay awake permanently in the background.
Newsstand downloads, remote notifications
Newsstand apps (as long as they are approved by Apple to be a major news organization and show up in the Newsstand part of the App Store) are permitted to download new content in the background; any app (like The New York Times app) that needs to display notifications generated outside the app (for example, breaking news notifications), can stay awake in the background to recieve such "remote" notifications.
Location Updates
If you allow an app to "always" use your location (i.e. in the background), then it can be active at any time in the background. Apps like Facebook, Find Friends, and others use this to update location-related information, though other apps can use it to surreptitiously use it to do tasks in the background (but only if you specifically allow them to access your location).
A very common misconception is that "swiping up" on an app to kill it stops it from operating in the background: this is wholly untrue. Apps that have one of the above seven reasons to operate in the background can operate in the background regardless of you "swiping up" on them, and apps that don't have one of these seven reasons can't operate when they're not on the screen even if you don't kill them.
Another misconception (that you seem to have!) is that switching off "Background App Refresh" will stop an app from operating in the background; in fact, that only disables an app from using Background Fetch, but it can operate in the background if it has one of the other six reasons too regardless of the Background App Refresh toggle.
So are the apps you have installed using CPU, memory, battery, or data in the background? You can easily find out! Go to Settings on your iPhone, find the Battery section, and take a peek. If an app shows "Background Activity" under its name, then it's been using CPU in the background; otherwise, it has not been doing anything in the background.
Here's an example screenshot:
On this iPhone, Podcasts has been operating in the background but none of the other apps have. Going into Battery Settings is a simple way to tell if Angry Birds has been running in the background.
So, to answer your questions directly:
When the iPhone restarts, can all these apps start up as well and take up valuable processor time and system resources?
Only if they are specifically configured to run in the background, you can check if they've been doing that in your iPhone's Battery settings.
What if I start an app, and then explicitly double tap on the Home button, and then slide the app out (to exit it). Can it or part of it still run in the background now or later?
Yes. For example, if Facebook uses location services (as it does), then swiping up to kill the Facebook app will not stop it from using your location in the background, and while it's using your location, it can perform just about any other task in the background too.
Since WhatsApp, WeChat, Line, and Facebook Messages all notify me of new messages, it looks like they will be running as soon as I restart my iPhone. Are the using my CPU in the background? Also, it seems that even if I turn off App Refresh in Settings, the apps can possibly still run in the background. Is that true?
Whatsapp, WeChat, Line, Hangouts, FB Messenger, and other chat apps use the remote notifications capability to check for new messages in the background every once in a while. This is expected behavior (and you can check how much they're doing it by checking their app-by-app battery usage, again in Battery settings), and shouldn't drain your battery too much. And indeed, turning off App Refresh for those apps will not have any effect, as the apps use remote notifications and not background fetch to operate in the background.
TL;DR: Apps can use your phone's resources in the background, you can see how much they're doing in Settings -> Battery.
Sources of information:
Best Answer
This is a known bug in iOS 11 and should be fixed with iOS 11.1 Most of the Apps are using the BackgroundFetch functionality, which is totally broken in iOS 11
Currently there is iOS 11.1 beta 4 released. It seems that Apple will release the final version within the delivery of the iPhone X.
If you want to have more technical details you can have a look at the technical topics in StackOverflow: iOS11 swift silent push (background fetch, didReceiveRemoteNotification) is not working anymore
UPDATE 2017-10-31
Just FYI: Apple released today iOS11.1 -> the issue should be now resolved from a operating system point of view.