June 9th, 2008
by Michael Swanberg
This is the second part in my series on getting more out of Windows Task Scheduler.
| ||||||||||
Last time, we talked about setting up a Windows version of the *nix built-in script Periodic. Today we’re going to discuss setting up scheduled tasks in a manner that doesn’t make everything seem convoluted and obscured. When getting into scheduling things to run periodically, it’s easy to get lost in the details, scheduling each job individually. That can tend to make the job lose some of its identity.
In essence, let’s say you want to run a backup weekly, and incremental backup daily at 9am and at 3pm, and a complete system backup monthly. If you have entries in your schedule that are labeled “Backup”, “IncBackup”, and “FullBackup” then there isn’t much there to reveal the nature of the schedule for these events.
On the other hand, if you have a backup job that’s sitting in a folder called “Weekly_Monday”, then it’s pretty obvious when this job will run. And once the scheduling for the “Weekly_Monday” folder of tasks is set up, you can pretty much forget it.
Okay, I’m not going to go too deep into the philosophy of all of this. Each person must figure out for herself what is important and what should be done. I am just going to go through how I have set up my recurring jobs and discuss why I did them that way, and you can judge for yourself if my methods are good, bad, ugly, or otherwise.
The first thing I did was to set up a folder where all my scheduled jobs could reside. This facilitates easy backups of the jobs, as well as a single location for all of them. I suggest if you have jobs that reside outside of this folder structure to set up a job within the Jobs folder that then executes your job that’s outside of it. This may be a little convoluted, but it will make things easier later when you have to ask yourself the question, “what is this and why did I set it up?”
Under the Jobs folder, I set up several subfolders, which should be somewhat obvious what their functions are. These folders are “Daily9am”, “DailyLate”, “DailyLunch”, “FirstMonday”, “Hourly”, “OnIdle”, and “OnLogon”. Notice that I refrained from using spaces in the names. This makes some things easier and avoids some errors that might be difficult to nail down.
Then I established a task in the TaskScheduler for each of these folders, executing Periodic on the particular folder. Here’s the Daily9am as an example: cmd /c start /MIN cmd /c Periodic.bat c:\Jobs\Daily9am
That seems like a lot of unnecessary garbage in there, but here’s how my thinking went. I want to run Periodic.bat with c:\Jobs\Daily9am as the parm. But I want it to run minimized so that it won’t pop up in front of whatever I am working on. So I need to use the start command with the “/MIN” option. Well, start is a command function (cmd), and not a program, so it has to be run in the cmd environment, hence the “cmd /c” preceding the “start”. The “/c” tells cmd to exit as soon as it’s done. If you wanted to see the results, you could use a “/k” which would tell cmd to stay open after it’s done.
Start defaults to “/k” behavior, so that is why the “cmd /c” is after the start. This causes the job to run and then exit when finished.
In every folder, I placed a small batch script with the following command:
echo Running... |perl -S log.pl >>Report.log
This is so that I can monitor the execution of each folder. Once I am satisfied that they’re all running when and how they should, I can remove these scripts.
You may be wondering what the log.pl script is. I’ll get to that in a second. First, let’s examine the command. The “echo Running…” prints “Running…” to the console. But the pipe “|” takes this output as standard input (StdIn) to the next command, which is the “perl -S log.pl” command. Finally, the “>>Report.log” appends the output to the file Report.log.
Now, what’s this log.pl, you are no doubt asking. Well, I wanted a way to have the date and time on each log entry. So I wrote a quick perl script to do this:
while (<>) {
print scalar(localtime),”\t$_”;
}
This essentially takes each line of StdIn, prepends the local time to it followed by a Tab character, and sends it to the Standard Output queue (StdOut). Very simple.
So why did I have to use the “perl -S” before the log.pl? Well, Windows doesn’t handle such things very well. I can, for instance, execute a perl program by simple calling the script, such as “log.pl”. But Windows doesn’t send the StdIn to it correctly. So I have to execute the Perl interpreter explicitly. The -S tells Perl to look on the standard path for the script. This way I don’t have to have the full path to the log.pl script in the command.
Finally, I did (what I think is) a clever thing with my Daily9am folder. In it, I placed a batch script called “Weeklies.bat” with the following code:
for /f %%f in ('today.pl fmt:w') do set dow=%%f
pushd .
dir /ad /b Weekly_* | egrep %dow% >dirs.txt
for /f %%f in (dirs.txt) do (
echo %%f
call Periodic.bat %%f
)
popd
del dirs.txt
This may be a little outside what most of you may wish to do, but I like it. Here’s the gist. Some things I want to run only on certain days of the week. Sure, I could set up schedules to do this explicitly, but this is a little better. Some things I want to run on, say, Monday and Thursday. Some things I want to run on Tuesday and Thursday. Rather than setting up different folders and different schedules for every combination I want to use, I created this script. Let’s look at it.
The first line, the “for” command is the only technique I have found for getting program output into batch variables. The for with the “/f” says that what’s in the parentheses is a program to be executed and the StdOut should be captured. “today.pl” is a perl script I set up that formats the date in pretty much whatever way I want to see it. In this case, I am interested in the day of the week, so I pass it the “fmt:w” parameter which tells the script to return only the 3-byte DOW (e.g. Mon, Tue, etc.). The set command takes what’s in %%f and places it in the dow variable. This has to be done this way because the %%f variable doesn’t have any life past the “for” command. It’s essentially the means to break that data out of that command.
The pushd and popd are housekeeping. There’s no telling what directory the “weekly” jobs will leave us in, so I want to return to where I was. Now, we set up the Periodic.bat script to take care of this for us, and it also runs each script in its own environment, so perhaps the pushd and popd are overkill, but better safe than sorry, right?
Next, I perform a little bit of voodoo. The “dir” command lists the contents of a folder, in this case the c:\Jobs\Daily9am folder. However, I tell it “/ad /b” which says “give me only directories, not files” and “give it to me in ‘brief’ format… just list them, don’t report on sizes, dates, total bytes, etc.”. Finally, I tell the dir command that I only want folders that start with “Weekly_” and end with anything else.
I take the output from that command and pipe it to the egrep program (which is a derivative of grep… either works fine). If you’re not familiar with grep, it is used mainly for finding text within files and other strings (StdIn). It then only outputs those lines of text that match the Regular Expression pattern. Regular Expressions is a topic of entire books, so I won’t go into it here. Just suffice it to say that I use the day of the week as returned from the today.pl script as the pattern. So now, the output is only those folders that start with “Weekly_” and also have the 3-byte day of the week in its name.
As such, I set up folders in the c:\Jobs\Daily9am folder and named them “Weekly_Fri”, “Weekly_Mon”, etc. I also set up folders called “Weekly_Mon_Thu” and “Weekly_Tue_Thu”. Notice, the list of folders that comes out of the egrep program will contain “Weekly_Mon_Thu” on both Mondays and Thursdays. Take care with these folder names. The today.pl outputs day names with the first letter capitalized. So the folders need to have them the same way for the egrep/grep to match on them. You could use “-i” in egrep/grep to make it ignore case, but that might introduce new issues. If you have a folder with “monster” or “wedding” in the name, these might execute on Mondays or Wednesdays. This is also why I started these folders explicitly with “Weekly_” so that there is nearly a zero percent chance of something getting executed by accident.
This list, by the way, gets pumped to a file: dir.txt. This file then gets read in by the next “for” command and each line gets stuffed into the %%f variable and then the following 2 lines are run with this value.
The “echo %%f” just displays the folder in question. Next, I execute Periodic.bat against the folder. So, say it’s Monday. The dirs.txt file will have 2 lines in it: “Weekly_Mon” and “Weekly_Mon_Thu”. So for these 2 folders, everything inside will get executed.
I know, it’s a little convoluted, but the point is, once everything is set up, all you have to do is drop job scripts and programs into the “Weekly_” folders to get them to execute on those days. And if you have something you want to run on multiple days, drop it in a folder that has all the days you want as part of the name. You could drop the job in multiple folders, but I don’t like to do it that way. Say I want to delete the job… if I miss it in one location, then it will still run. It’s best to have only one script file in one location for a given scheduled function.
Finally, I have one other oddball that’s worth noting. I have a mainframe system that runs on weeknights. After it is finished, I have some things I like to run on my PC. So I set up a job to determine if the mainframe jobs are finished (I won’t go into the details of this), and if so, it runs Periodic against a folder full of tasks.
All in all, this whole process is designed to keep me from having to clutter up my Scheduled Tasks list with individual things. With a handful of schedules that are static, I can do pretty much anything I need to. And any changes are a snap: I just move the script to another folder, delete it, drop in new scripts, whatever I want. And I don’t have to fiddle with the Task Scheduler.
This also takes the details of the task scheduler out of the schedule. If you use other software for running jobs at certain times, this process works the same way, once the schedule is established.
No muss, no fuss! That’s my motto.
Related Posts on This Topic:
Taming the Schedule, Part 1
What I Would Like To See…
The Google Grace Period for New Websites
Intel Inside… Your TV???
What is Podcasting? (part 1)
