job-collection for Meteor.js

Job collections are backed by Meteor Collections and support standard reactive methods like .find() and .findOne() on the client.

Important Security Note: Meteor clients are automatically denied permission to directly insert, update, or remove jobs. To modify jobs, you must use the provided JobCollection, Job, and JobQueue APIs. Servers retain access to standard Meteor collection methods but should favor the job collection APIs to ensure the job document model is correctly maintained and enforced.

For non-Meteor clients, you can use the meteor-job npm package to implement identical functionality via the same interfaces.

You can add custom, private server-side data to a job document by using a subdocument named _private.

Important Security Warning: Data inside _private is not accepted via or returned from standard jobCollection method calls. However, if you include _private in a query cursor returned via a Meteor publish function, you will leak this sensitive data to clients. Always ensure you exclude _private from any published cursors.

The job-collection package is built on top of MongoDB and uses Meteor's DDP protocol for persistence, reactivity, and secure remote access.

Key architectural points:

• JobCollection as a Collection: A JobCollection is a standard Meteor Collection. While you can use .find() and .findOne(), most interactions happen through the Job object API.
• Method-based API: Most Job API calls are transformed into Meteor Methods. This allows the Job class to be implemented in pure JavaScript, making it compatible with Meteor servers, Meteor clients, and independent Node.js processes (via the meteor-job package).

Each job-collection instance created on a Meteor server defines a set of DDP methods. These methods are prefixed with the name of the job collection (e.g., myJobs_getWork) to prevent collisions between multiple collections on the same server.

While the JobCollection and Job APIs are the recommended way to interact with the library, you can call these DDP methods directly if you require finer control over allow/deny rules than the predefined admin, manager, creator, and worker access categories provide.

Securing a job-collection follows standard Meteor patterns (publish/subscribe and allow/deny rules) but includes specialized remote methods.

To manage permissions efficiently, you can write allow/deny rules for one of the four predefined permission groups:

• admin
• manager
• creator
• worker

These roles group related functions to separate security concerns. If these predefined roles do not meet your requirements, you can secure each remote method individually with custom allow/deny rules.

To add job-collection to your Meteor application, use the following command. This package exposes a global JobCollection object on both the client and the server.

meteor add vsivsi:job-collection

Completed and canceled jobs accumulate in the database by default. To prevent database bloat, use one of the following strategies:

1. Add a job cleaning job: Create a recurring job specifically designed to clean up old records based on your custom logic.
2. Use events: Listen to jc.events to remove jobs automatically once they complete or are removed.

To use job-collection, you first initialize a JobCollection on the server. You can use .allow() to define security rules for which users can perform actions on the jobs. Finally, call .startJobServer() within Meteor.startup to begin processing the queue.

///////////////////
// Server
if (Meteor.isServer) {

  var myJobs = JobCollection('myJobQueue');
  myJobs.allow({
    // Grant full permission to any authenticated user
    admin: function (userId, method, params) {
      return (userId ? true : false);
    }
  });

  Meteor.startup(function () {
    // Normal Meteor publish call, the server always
    // controls what each client can see
    Meteor.publish('allJobs', function () {
      return myJobs.find({});
    });

    // Start the myJobs queue running
    return myJobs.startJobServer();
  });
}

To avoid the latency of time-based polling, you can set a very high pollInterval and use Meteor's observe to trigger the queue whenever a new job is added to the collection.

Note: For non-Meteor Node.js worker scripts, use the alternative approach documented in the meteor-job npm package.

// 1. Create a queue that doesn't poll automatically
const q = jc.processJobs(
  'jobType',
  {
    pollInterval: 1000000000, // Effectively disable polling
  },
  function (job, callback) {
    job.done();
    callback();
  }
);

// 2. Use Meteor observe to trigger the queue when a job is added
jc.find({ type: 'jobType', status: 'ready' })
  .observe({
     added: function () { q.trigger(); }
  });