Short Answer
It looks like the data in msdb.dbo.sysjobschedules is updated by a background thread in SQL Agent, identified as SQLAgent - Schedule Saver, every 20 minutes (or less frequently, if xp_sqlagent_notify has not been called and no jobs have run in the meantime).
For more accurate information, look at next_scheduled_run_date in msdb.dbo.sysjobactivity. This is updated in real-time any time a job is changed or a job has run. As an added bonus, the sysjobactivity stores the data the right way (as a datetime column), making it a lot easier to work with than those stupid INTs.
That's the short answer:
It could be up to 20 minutes before sysjobschedules reflects the truth; however, sysjobactivity will always be up to date. If you want a lot more details about this, or how I figured it out...
Long Answer
If you care to follow the rabbit for a moment, when you call sp_add_jobschedule, this chain of events is set into motion:
msdb.dbo.sp_add_jobschedule == calls ==> msdb.dbo.sp_add_schedule
msdb.dbo.sp_attach_schedule
msdb.dbo.sp_attach_schedule == calls ==> msdb.dbo.sp_sqlagent_notify
msdb.dbo.sp_sqlagent_notify == calls ==> msdb.dbo.xp_sqlagent_notify
Now, we can't chase the rabbit any further, because we can't really peek into what xp_sqlagent_notify does. But I think we can presume that this extended procedure interacts with the Agent service and tells it that there has been a change to this specific job and schedule. By running a server-side trace we can see that, immediately, the following dynamic SQL is called by SQL Agent:
exec sp_executesql N'DECLARE @nextScheduledRunDate DATETIME
SET @nextScheduledRunDate = msdb.dbo.agent_datetime(@P1, @P2)
UPDATE msdb.dbo.sysjobactivity
SET next_scheduled_run_date = @nextScheduledRunDate
WHERE session_id = @P3 AND job_id = @P4',
N'@P1 int,@P2 int,@P3 int,@P4 uniqueidentifier',
20120819,181600,5,'36924B24-9706-4FD7-8B3A-1F9F0BECB52C'
It seems that sysjobactivity is updated immediately, and sysjobschedules is only updated on a schedule. If we change the new schedule to be once a day, e.g.
@freq_type=4,
@freq_interval=1,
@freq_subday_type=1,
@freq_subday_interval=0,
@freq_relative_interval=0,
@freq_recurrence_factor=1,
We still see the immediate update to sysjobactivity as above, and then another update after the job is finished. Various updates come from background and other threads within SQL Agent, e.g.:
SQLAgent - Job Manager
SQLAgent - Update job activity
SQLAgent - Job invocation engine
SQLAgent - Schedule Saver
A background thread (the "Schedule Saver" thread) eventually comes around and updates sysjobschedules; from my initial investigation it appears this is every 20 minutes, and only happens if xp_sqlagent_notify has been called due to a change made to a job since the last time it ran (I did not perform any further testing to see what happens if one job has been changed and another has been run, if the "Schedule Saver" thread updates both - I suspect it must, but will leave that as an exercise to the reader).
I am not sure if the 20-minute cycle is offset from when SQL Agent starts, or from midnight, or from something machine-specific. On two different instances on the same physical server, the "Schedule Saver" thread updated sysjobschedules, on both instances, at almost the exact same time - 18:31:37 & 18:51:37 on one, and 18:31:39 & 18:51:39 on the other. I did not start SQL Server Agent at the same time on these servers, but there is a remote possibility that the start times happened to be 20 minutes offset. I doubt it, but I don't have time right now to confirm by restarting Agent on one of them and waiting for more updates to happen.
I know who did it, and when it happened, because I placed a trigger there and captured it, in case I couldn't find it in the trace, or I inadvertently filtered it out.
CREATE TABLE dbo.JobAudit
(
[action] CHAR(1),
[table] CHAR(1),
hostname SYSNAME NOT NULL DEFAULT HOST_NAME(),
appname SYSNAME NOT NULL DEFAULT PROGRAM_NAME(),
dt DATETIME2 NOT NULL DEFAULT SYSDATETIME()
);
CREATE TRIGGER dbo.schedule1 ON dbo.sysjobactivity FOR INSERT
AS
INSERT dbo.JobAudit([action], [table] SELECT 'I', 'A';
GO
CREATE TRIGGER dbo.schedule2 ON dbo.sysjobactivity FOR UPDATE
AS
INSERT dbo.JobAudit([action], [table] SELECT 'U', 'A';
GO
CREATE TRIGGER dbo.schedule3 ON dbo.sysjobschedules FOR INSERT
AS
INSERT dbo.JobAudit([action], [table] SELECT 'I', 'S';
GO
CREATE TRIGGER dbo.schedule4 ON dbo.sysjobschedules FOR UPDATE
AS
INSERT dbo.JobAudit([action], [table] SELECT 'U', 'S';
GO
That said, it is not hard to catch with a standard trace, this one even comes through as non-dynamic DML:
UPDATE msdb.dbo.sysjobschedules
SET next_run_date = 20120817,
next_run_time = 20000
WHERE (job_id = 0xB87B329BFBF7BA40B30D9B27E0B120DE
and schedule_id = 8)
If you want to run a more filtered trace to track this behavior over time (e.g. persisting through SQL Agent restarts instead of on-demand), you can run one that has appname = 'SQLAgent - Schedule Saver'...
So I think that if you want to know the next run time immediately, look at sysjobactivity, not sysjobschedules. This table is directly updated by Agent or its background threads ("Update job activity", "Job Manager" and "Job invocation engine") as activity happens or as it is notified by xp_sqlagent_notify.
Be aware, though, that it is very easy to muck up either table - since there are no protections against deleting data from these tables. (So if you decided to clean up, for example, you can easily remove all the rows for that job from the activity table.) In this case I'm not exactly sure how SQL Server Agent gets or saves the next run date. Perhaps worthy of more investigation at a later date when I have some free time...
Best Answer
This is best done outside of SQL Agent. Honestly, at 800+ jobs running every hour you should have moved to a different method long before now.
My preference for this would be to create a single master SSIS package that reads in a list of jobs that need to be run and then starts up the first X in the list, then sleeps until the number of running jobs is below threshold, then starts up the next X in the list. You could randomize the list or sort it, as you choose. I've done something similar to this in my environment where I needed to make sure that certain classes of packages (client imports) were done one at a time.
The less dynamic approach would be to just break the list of jobs that need to be run into however many simultaneous you want running. Each of those would have a list of steps that call the actual job that needs running. It takes less fiddling about with timing, sleeps and querying active jobs but is less dynamic (for example, changing the number of max simultaneous threads will take work.
But in any case, I would leave the jobs as they are now, just remove/disable their schedules. Then whatever solution you want can just call the sp_start_job procedure. This lets you still easily run any job you want out of cycle and you see independent reports of success/failure.