- Jan 2021
I started using Cronicle a few weeks ago and really like it. Runs on a server... https://github.com/jhuckaby/Cronicle
This also ticks a lot of my desired features.
Really easy to set up if you already have node ready to go.
UI is very slick and feels right to me out of the box.
Jobs can be assigned to categories. A given category can have max concurrent processes running at the same time (so run 1 backup task a time, even though 5 tasks are scheduled within the same time period). Individual tasks can also be set to be singleton or configurable max concurrency.
Supports configurable retry (number of attempts, delay between).
Supports optional catchup runs if runs are missed or queued runs.
Supports killing and erroring out if timeouts or resource limits are hit.
Time from download to first job setup... 2 minutes? Very intuitive UI.
Has management API, not clear if it has an existing good CLI interface.
Also supports setting up users to be able to run pre-defined scripts and see output.
Need to figure out how to back-up and restore jobs.
Very quick impression is this ticks a lot of my desired features.
I'm not wild about the community edition default dashboard - I'd rather a more high level view of everything configured and its statuses.
UI is clunky when compared to Cronicle. Lots of steps to get from setting it up to actually running something. No quick click from a run task to its log output. No good resources/stats view that I found.
Like the fact it keeps track of logs, runtime (and can alert if runtime deviates from normal), gives you an estimated time to complete, lets you run on a schedule and/or manually.
Like the fact it supports farming tasks off through SSH or other, or running them locally. Can auto-discover nodes using a script you provide (e.g. query AWS nodes) or using static config.
Really interested in the multi-user capabilities. This may solve a problem I didn't really know I had at work (giving a semi-technical person access to kick off jobs or monitor them before asking me).
Running all that manually (more than 100 scripts across all devices) is an awful job for a human. I want to set them up once and more or less forget about it, only checking now and then.
My ideals for all of my regular processes and servers:
- Centralized configuration and control - I want to go into a folder and configure everything I'm running everywhere.
- Configuration file has the steps needed to set up from scratch - so I can just back up the configuration and data folders and not worry about backing up the programs.
- Control multiple machines from the central location. Dictate where tasks can run.
- [nice to have] Allow certain tasks to running externally, e.g. in AWS ECS or Lambda or similar
- Command-line access for management (web is great for monitoring)
- Flexible scheduling (from strict every minute to ~daily)
- Support for daemons, psuedo-daemons (just run repeatedly with small delays), and periodic tasks.
- Smart alerts - some processes can fail occasionally, but needs to run at least once per day - some processes should never fail. A repeating inaccurate alert is usually just as bad as no alert at all.
- Error code respect (configurable)
- Logs - store the program output, organize it, keep it probably in a date-based structure
- Health checks - if it's a web server, is it still responding to requests? Has it logged something recently? Touched a database file? If not, it's probably dead.
- Alerts support in Telegram and email
- Monitor details about the run - how long did it take? How much CPU did it use? Has it gotten slower over time?
- Dashboard - top-level stats, browse detailed run stats and logs
So much of the configuration/control stuff screams containers, so more and more I'm using Docker for my scripts, even simpler ones.
I'm pretty sure a lot of this is accomplished by existing Docker orchestration tools. Been delaying that rabbit hole for a long time.
I think the key thing that makes this not just a "cron" problem for me, is I want something that monitors and manages both itself and the tasks I want to run, including creating/setting up if not already. I also want to ideally focus my mental energy into a single controller that handles my "keep this running" things all together, be they servers or infrequent tasks.
Doesn't have to be a single project. Might be multiple pieces glued together somehow.
- Oct 2020
- Jan 2016
Assignments and Architecture: Pedagogy in the Digital Age