How to Implement Task Scheduling with JobRunr and Markdown Files in Spring Boot

Mar 25, 2025

I was working on a background task system and ran into a familiar problem: I couldn’t see what tasks were running without querying the database.

Not just me — my teammates couldn’t either. We’d SSH into the server, run SQL queries, and squint at timestamps. Something had to change.

The Problem with Traditional Task Scheduling

Most task schedulers store job state in a database. This works, but it creates friction:

Tasks are invisible without admin tools
Cron expressions are cryptic (0 0 9 * * ? — what does that even mean?)
Debugging requires database queries
Auditing task history is painful
Non-developers can’t view or modify tasks

I wanted something more transparent. Something I could cat or ls to understand what’s happening.

The Idea: Markdown Files as Tasks

What if each task was a Markdown file?

workspace/tasks/2025-03-25/091500-todo-send-report.md

I could see all pending tasks with find. I could read any task with a text editor. I could even version control them.

But I still needed robust scheduling — retries, delays, recurring jobs. That’s where JobRunr comes in.

Setting Up JobRunr

JobRunr is a Java background job processor. It handles persistence, retries, and gives you a dashboard out of the box.

First, I added the dependency:

<dependency>
    <groupId>org.jobrunr</groupId>
    <artifactId>jobrunr-spring-boot-3-starter</artifactId>
    <version>8.5.0</version>
</dependency>

Then configured it:

jobrunr:
  dashboard:
    port: 8081
  background-job-server:
    worker-count: 1

The dashboard at http://localhost:8081 shows all jobs, their status, and history. Super useful for debugging.

Task File Structure

I decided on a naming convention that encodes state:

yyyy-MM-dd/HHmmss-<state>-<name>.md

For example:

workspace/
└── tasks/
    ├── 2025-03-25/
    │   ├── 091500-todo-send-daily-report.md
    │   ├── 092000-in_progress-backup-database.md
    │   └── 093000-completed-generate-invoice.md
    └── recurring/
        └── daily-backup.md

The state is right in the filename. ls shows me everything:

$ ls workspace/tasks/2025-03-25/
091500-todo-send-daily-report.md
092000-in_progress-backup-database.md
093000-completed-generate-invoice.md

Task File Format

Each task file is a Markdown document with YAML frontmatter:

---
id: task-2025-03-25-091500
state: todo
scheduled: 2025-03-25T09:15:00
created: 2025-03-25T09:00:00
---

# Send Daily Report

## Description
Generate and send the daily activity report to stakeholders.

## Parameters
- report_type: daily_summary
- recipients:
  - [email protected]
  - [email protected]

This is human-readable. I can open it in any text editor.

The State Machine

Tasks transition through states:

todo → in_progress → completed
                   → awaiting_human_input

I made sure to update the filename when the state changes. This keeps the filesystem in sync with JobRunr’s internal state.

Integrating with JobRunr

Now for the fun part — connecting Markdown files to JobRunr.

One-off Tasks

For immediate execution:

@Service
public class TaskSchedulerService {

    @Inject
    private TaskRepository taskRepository;

    public void scheduleTask(Task task) {
        // Save as Markdown file
        taskRepository.save(task);

        // Enqueue with JobRunr
        BackgroundJob.enqueue(() -> executeTask(task.getId()));
    }
}

Delayed Tasks

For scheduling in the future:

case DELAYED:
    BackgroundJob.schedule(
        task.getScheduledTime(),
        () -> executeTask(task.getId())
    );
    break;

Recurring Tasks

For cron-scheduled jobs, I created templates in tasks/recurring/:

---
cron: "0 0 9 * * ?"
type: recurring
---

# Daily Database Backup

Every day at 9:00 AM:
1. Create database snapshot
2. Compress backup file
3. Upload to cloud storage

Then register with JobRunr:

case RECURRING:
    RecurringJob.create(
        task.getId(),
        task.getCronExpression(),
        () -> executeTask(task.getId())
    );
    break;

The Execute Method

Here’s where the state transitions happen:

@Job(name = "Execute task %0")
public void executeTask(String taskId) {
    Task task = taskRepository.findById(taskId);

    // Update state to in_progress
    task.setState(TaskState.IN_PROGRESS);
    taskRepository.save(task);  // Updates file and filename

    try {
        taskExecutor.execute(task);
        task.setState(TaskState.COMPLETED);
    } catch (Exception e) {
        task.setState(TaskState.AWAITING_HUMAN_INPUT);
        task.setError(e.getMessage());
    }

    taskRepository.save(task);
}

The Task Repository

I wrote a repository that reads and writes Markdown files:

@Repository
public class TaskRepository {

    public Task findById(String taskId) {
        Path taskFile = findTaskFile(taskId);
        return parseMarkdown(taskFile);
    }

    public void save(Task task) {
        Path oldPath = findTaskFile(task.getId());
        Path newPath = buildPath(task);  // Includes state in filename

        if (!oldPath.equals(newPath)) {
            Files.move(oldPath, newPath);  // Rename if state changed
        }

        writeMarkdown(newPath, task);
    }

    private Task parseMarkdown(Path file) {
        String content = Files.readString(file);
        // Parse YAML frontmatter and Markdown body
        return TaskParser.parse(content);
    }
}

The key insight: renaming the file when state changes keeps everything discoverable.

What I Learned

JobRunr handles the hard stuff

Persistence (uses a database internally for job state)
Retries with exponential backoff
Dashboard for monitoring
Distributed processing if needed

Markdown files handle visibility

Anyone can read a .md file
find and grep work out of the box
Version control is trivial
Non-developers can understand the structure

The combination works well

JobRunr manages execution. Markdown files manage transparency. They complement each other.

Common Mistakes

I made these mistakes so you don’t have to:

1. Forgetting to update filenames on state change

If the state changes but the filename doesn’t, ls shows wrong information. Always rename.

2. Setting worker-count too high

I set worker-count: 10 on a small server and ran out of memory. Start with 1 or 2.

3. Not checking the JobRunr dashboard

When jobs fail, the dashboard shows exactly why. Don’t guess — look at the dashboard.

4. Cron expressions without documentation

0 0 9 * * ? is meaningless without context. I started adding comments:

## Schedule
Every day at 9:00 AM (cron: 0 0 9 * * ?)

Summary

JobRunr + Markdown files = transparent, auditable task scheduling.

JobRunr handles background job processing, retries, and monitoring
Markdown files make tasks visible and version-controllable
Filename convention (HHmmss-<state>-<name>.md) makes state visible at a glance
Dashboard at port 8081 provides job monitoring

The JavaClaw project uses this pattern. It works well for teams that want visibility into their background jobs without building admin tools.

Final Words + More Resources

My intention with this article was to help others share my knowledge and experience. If you want to contact me, you can contact by email: Email me

Here are also the most important links from this article along with some further resources that will help you in this scope:

👨‍💻 JobRunr Documentation
👨‍💻 Spring Boot Scheduling

Oh, and if you found these resources useful, don’t forget to support me by starring the repo on GitHub!