1

Background

I work at a research institute, and have for a long time used the batch command to submit job queues to our machines. However, a recent update changed the batch command to be POSIX-compliant, the result of which is it doesn't take input files anymore. Thus, I would have to manually enter each command at the at> prompt, rather than reusing an old queue input file and making minor changes where needed. This is rather time-consuming, and considering how terribly batch handles jobs anyway, is almost a waste of time as opposed to just starting the jobs myself.

The Problem

The only replacement queues I have been able to find are either prohibitively expensive, or don't do the job. I am looking for a queue that simply runs the jobs it's given in temporal order; no priority queues or anything like that. What queues are available now that batch doesn't function the same way?

Jonathan E. Landrum
  • 227
  • 4
  • 15
  • To be fair, I can accomplish something that is similar to the old batch method using at now < queue_list. – Jonathan E. Landrum Oct 24 '14 at 20:49
  • Have you tried expect (or pexpect)? – John1024 Oct 24 '14 at 21:04
  • @John1024 that looks like a neat project, but it's not exactly what a queue accomplishes. A proper job queue starts the next process in line when the amount of resources requested becomes available. More elaborate queues search the waiting list for jobs that best fit the amount of resources available. It's more than just automation. It's resource management. – Jonathan E. Landrum Oct 24 '14 at 21:10
  • 1
    I was suggesting that you use expect to enter the information that you need at your batch prompt. batch would be what handles the queue. What expect provides is a way to automatically enter programmed information at a batch prompt. – John1024 Oct 24 '14 at 21:17
  • I see, now that would work. I completely missed your objective. – Jonathan E. Landrum Oct 24 '14 at 21:23

1 Answers1

2

expect (and pexpect) were designed to automate interaction with programs that want "interactive" input. expect allows one to automate starting a program, waiting for its prompt, sending a response, waiting for another prompt, etc.

Here is a simple example of an expect script that starts batch and creates a job for it to schedule. It starts batch, waits for a prompt, creates a job, waits for a prompt, sends a control-D (\004), waits for batch to respond with confirming job creation, such as job 271 at Sat Oct 25 14:31:00 2014, and then waits a second for batch to complete and exits:

#!/usr/bin/expect --

spawn batch
expect "at>"
send "echo Hello from batch\r"
expect "at>"
send "\004"
expect "job"
sleep 1

expect has many advanced features. You can create procedures and manipulate variables. For more information, see the expect homepage and the expect FAQ. The expect wikipedia page is also very informative.

I haven't tried it but it might be possible to create an expect script that reads from stdin and queues jobs. Alternatively, one might create a shell/sed/awk/python script to read from stdin on input and write an expect script on output.

John1024
  • 74,655