A repository of tips and tricks (in both English and French) curated by Mirego’s engineering team.
  • applescript
  • jxa

Automatic timesheets with Timing and Harvest

At Mirego, we're using Harvest to track time on various projects. It allows every team member to report the exact time spent on various projects for our different clients, and this data can have many purposes, like invoicing and reporting.

Today I learned how to make this reporting almost entirely seamless for anyone using a Mac computer, along with the Timing app and some basic scripting.

Time tracking with Harvest

Harvest comes with a great Mac application, in which you can start timers and assign your time to the project that you are currently working on.

While the tool is great, the process works best without too many interruptions. When things start stacking on each other, it may be harder to keep up, as there can be some heavy context-switching between several projects.

An alternative to this timer is to enter entries manually in retrospect, at the end of each day or week. This process can be helped by looking at things like your calendar events, browsing history and even phone call logs, but you may always end up forgetting about things, and the operation may be rather intensive.

For people like me, the ideal approach would be some hybrid between these two – and this is exactly where the Timing app comings into play.

Auto-tracking with Timing

With the marvelous Timing app for macOS, time spent across several projects can be tracked automatically, at any given time of the day.

The app is very simple to setup and does all the work magically, by trying to identify what you have been doing on your computer in real-time.

Each project in Timing is configured with a πŸ“› Name, πŸ”΅ Color and πŸ“Β TrackingΒ rules. Start by creating these for every different "time bucket" that you want to report, and let the magic happens.

Here are a few tips based on several months of use, to make the most out of it:

  • 1️⃣ 1:1 Match

    • Use one Timing project for every Harvest task
    • This will then make it easy to submit time entries, as it's a 1:1 match
  • πŸ—“ Calendar Integration

    • Attach your work calendar to Timing, to see all your events on the report timeline
    • It will then be very simple to add time in respective projects for these timeslots (especially if you become inactive, or take the call away from your computer)
  • πŸ‹ Codenames & Emojis

    • When feasible, assign code names and emojis to identify every project you work on
    • Make sure to add either of those (or both!) to anything that refers to this project across different services (e.g. by prefixing calendar event title, renaming to-do lists, inbox labelling, etc)
    • This will allow to create simple rules for auto-tracking, by matching on these keyworks instantly
  • 🏷 Exhaustive Tagging

    • Figure out anything else that is unique around a specific project (e.g. the people you work with, the services you're using, for things like project management or video meetings, etc)
    • Create rules around these, and don't hesitate to try different things on different days – trail and error is the best way to eventually prevent errors
  • πŸ”„ Frequent Reviews

    • Don't wait too long before reviewing your reports – at least a daily checkup ensures that none of the rules went crazy and all apps were correctly captured
    • The sooner you review, the easier it will be to remember what you were doing – but don't worry, the whole purpose of this app is to remember that for you, so you should always be able to figure it out just by looking at the timeline

After playing with the rules and balancing the reports for a couple of days, the time spent in different apps and documents will be mostly auto-assigned to the configured projects.

The reports from Timing can then be used as a reference to submit time entries into Harvest manually.

Timing Report Harvest Entries
Timing Report Harvest Entries

This means you can now context-switch however you wish: all your work will be reported straight into the right project, without having to switch your Harvest timers. πŸŽ‰

Auto-publishing with AppleScript

If getting there already sounded great, the ultimate setup would be to have Timing reports automatically published to Harvest, without any further intervention. Luckily for us, Timing provides an AppleScript interface and Harvest provides a developer API.

Using JavaScript AppleScript (JXA), the first step is to configure a mapping between Timing projects and Harvest tasks:

{
  "projects": [
    // Client projects
    { "name": "πŸ‹ Citrus", "code": "CITRUS-001", "task": "Project Management", "defaultNote": "Follow-up" },
    { "name": "πŸ“ Berry", "code": "BERRY-001", "task": "Project Management", "defaultNote": "Follow-up" },

    // Internal projects
    {"name": "Writing", "code": "INTERNAL", "task": "Writing", "defaultNote": "" },
    {"name": "Open-Source", "code": "INTERNAL", "task": "Open-Source", "defaultNote": "" },

    // Fallback
    {"name": "[FALLBACK]", "code": "INTERNAL", "task": "Other" }
  ]
}

Then the following steps can be accomplished with scripting (see full script below):

  • Export report from Timing using its AppleScript interface
  • Process report and assign time entries to configured Harvest project
  • Publish entries into Harvest using the developer API

Which provides the following output:

Script Output Harvest Entries
Script Output Harvest Entries

This means you can now context-switch without even having to worry about time reporting: everything is automatically tracked and pushed to Harvest. πŸŽ‰

References

The script

Script usage

timing-harvest.jxa publishingMode reportingMode [timeframe=today]

Using the following parameters:

  • publishingMode

    • read: Only process report (do not submit to Harvest)
    • write: Process report and submit to Harvest
  • reportingMode

    • draft: Prepends every note with a draft message
    • final: Sends all notes without change
  • [timeframe=today]

    • today: Process report for current date
    • yesterday: Process report for previous date
    • lastWeek: Process report for last week (Sat – Fri)
    • thisWeek: Process report for this week (Sat – Fri)
    • date1,date2: Process timeframe (format: yyyy-mm-dd)

Running the script

To make sure the script can be run directly from the command-line, run chmod +x on the script file and run it based on the usage described above.

Programming the script

When the script has been correctly configured, setup a cron task on your Mac to automatically run this script every 30 minutes. Run crontab -e and enter a configuration like this one:

0,30 * * * * /path/to/timing-harvest.jxa write draft today

Full source code

#!/usr/bin/env osascript -l JavaScript

// Default parameters
let DEFAULT_TIMEFRAME = 'today';
let DEFAULT_IS_DRAFT = false;
let DEFAULT_APPLY_CHANGES = false;

// App config
var config = {
  harvest: {
    // Retrieve from here: https://id.getharvest.com/developers
    account: '11111',
    token: '11111.pt.abcdefghijklmnopqrstuvwxyz',
    agent: 'John Doe (johndoe@me.com)'
  },
  options: {
    // Path of the log file
    logFilePath: '/path/to/logging.log',
    // Text prefix for notes when sending in "Draft" mode
    draftPrefix: '⚠️ DRAFT ⚠️',
    // Max number of lines, when adding notes to time entries
    maxDescriptionTasks: 4,
    // Minimum duration of time entries to keep them in report (fraction of hours)
    minimumEntryLength: 5 / 60,
    // Rounding factor to apply on time entries (fraction of hours)
    hourRoundUp: 1 / 1000
  },
  projects: [
    // Client projects
    {
      name: 'πŸ‹ Citrus',
      code: 'CITRUS-001',
      task: 'Project Management',
      defaultNote: 'Follow-up'
    },
    {
      name: 'πŸ“ Berry',
      code: 'BERRY-001',
      task: 'Project Management',
      defaultNote: 'Follow-up'
    },

    // Internal projects
    {name: 'Writing', code: 'INTERNAL', task: 'Writing', defaultNote: ''},
    {
      name: 'Open-Source',
      code: 'INTERNAL',
      task: 'Open-Source',
      defaultNote: ''
    },

    // Fallback
    {name: '[FALLBACK]', code: 'INTERNAL', task: 'Other'}
  ]
};

//////////////////////////////////
//         INITIALIZERS         //
//////////////////////////////////

ObjC.import('Foundation');
var app = Application.currentApplication();
app.includeStandardAdditions = true;

//////////////////////////////////
//          MAIN SCRIPT         //
//////////////////////////////////

function run(argv) {
  try {
    // Parse parameters
    if ([0, 2, 3].indexOf(argv.length) == -1) {
      appLog('Missing parameters. Suggested usage:');
      appLog(
        '  timing-harvest.jxa publishingMode reportingMode [timeframe=today]'
      );
      appLog('');
      appLog('  publishingMode');
      appLog('   read:         Only process report (do not submit to Harvest)');
      appLog('   write:        Process report and submit to Harvest');
      appLog('');
      appLog('  reportingMode');
      appLog('   draft:        Prepends every note with a draft message');
      appLog('   final:        Sends all notes without change');
      appLog('');
      appLog('  [timeframe=today]');
      appLog('   today:        Process report for current date');
      appLog('   yesterday:    Process report for previous date');
      appLog('   lastWeek:     Process report for last week (Sat – Fri)');
      appLog('   thisWeek:     Process report for this week (Sat – Fri)');
      appLog('   date1,date2:  Process timeframe (format: yyyy-mm-dd)');
      return;
    }

    var applyChanges = DEFAULT_APPLY_CHANGES;
    var isDraft = DEFAULT_IS_DRAFT;
    var timeframe = DEFAULT_TIMEFRAME;

    if (argv[0] && argv[1]) {
      applyChanges = argv[0] == 'write';
      isDraft = argv[1] == 'draft';
    }

    if (argv[2]) {
      timeframe = argv[2];
    }

    // Parse timeframe – default to today
    var startDate = removeTime(new Date());
    var endDate = removeTime(new Date());

    if (timeframe.match(/\d{4}-\d{2}-\d{2},\d{4}-\d{2}-\d{2}/)) {
      // Timespan (e.g. 2020-01-01,2020-01-07)
      let components = timeframe.split(',');
      startDate = parseDate(components[0]);
      endDate = parseDate(components[1]);
    } else if (timeframe.match(/\d{4}-\d{2}-\d{2}/)) {
      // Single date (e.g. 2020-01-01)
      startDate = parseDate(timeframe);
      endDate = parseDate(timeframe);
    } else if (timeframe == 'lastWeek') {
      // Saturday to Friday from the previous week
      let variation = startDate.getDate() - ((startDate.getDay() + 1) % 7) - 7;
      startDate.setDate(variation);
      endDate.setDate(variation + 6);
    } else if (timeframe == 'thisWeek') {
      // Saturday to Friday from the current week
      let variation = startDate.getDate() - ((startDate.getDay() + 1) % 7);
      startDate.setDate(variation);
      endDate.setDate(variation + 6);
    } else if (timeframe == 'yesterday') {
      // 1 day ago
      [startDate, endDate].forEach((d) => d.setDate(d.getDate() - 1));
    }

    // Log header
    appLog('=====================================================');
    appLog('Processing Harvest Timesheets');
    appLog('=====================================================');
    appLog(`Publishing Mode: ${applyChanges ? '✏️ Write' : 'πŸ‘€ Read-only'}`);
    appLog(`Reporting Mode: ${isDraft ? '⚠️ Draft' : 'βœ… Final'}`);
    if (startDate.getDay() == 6 && endDate.getDay() == 5) {
      var displayStartDate = new Date(startDate.getTime());
      displayStartDate.setDate(displayStartDate.getDate() + 2);
      appLog(
        `Work Week: ${[displayStartDate, endDate]
          .map((d) => d.toDateString())
          .join(' - ')}`
      );
    } else if (startDate.getTime() != endDate.getTime()) {
      appLog(
        `Dates: ${[startDate, endDate]
          .map((d) => d.toDateString())
          .join(' - ')}`
      );
    } else {
      appLog(`Date: ${startDate.toDateString()}`);
    }
    appLog('=====================================================');
    appLog('');

    // Process report
    var report = timingReport(startDate, endDate);
    var projectsByDay = processReport(report, isDraft);

    // Submit report
    if (applyChanges) {
      submitToHarvest(projectsByDay);
    }

    appLog('Process complete.');
  } catch (err) {
    appLog(err);
    appLog('Process aborted.');
    displayNotification(err.toString());
  }
}

//////////////////////////////////
//      PROCESS TIMING DATA     //
//////////////////////////////////

function timingReport(startDate, endDate) {
  appLog('Exporting entries from Timing app...');

  var timingHelper = Application('TimingHelper');

  // Report settings
  var reportSettings = timingHelper.ReportSettings().make();

  reportSettings.firstGroupingMode = 'by project';
  reportSettings.secondGroupingMode = 'by day';

  reportSettings.tasksIncluded = true;
  reportSettings.taskTitleIncluded = true;
  reportSettings.taskTimespanIncluded = true;
  reportSettings.taskNotesIncluded = true;

  reportSettings.appUsageIncluded = true;
  reportSettings.applicationInfoIncluded = true;
  reportSettings.titleInfoIncluded = true;
  reportSettings.timespanInfoIncluded = true;
  reportSettings.includeAppActivitiesCoveredByATask = false;

  // Export settings
  var exportSettings = timingHelper.ExportSettings().make();

  exportSettings.fileFormat = 'JSON';
  exportSettings.durationFormat = 'hours';
  exportSettings.shortEntriesIncluded = true;

  let exportTempPath = $.NSTemporaryDirectory().js + 'harvest.json';

  // Process
  timingHelper.saveReport({
    withReportSettings: reportSettings,
    exportSettings: exportSettings,
    between: startDate,
    and: endDate,
    to: exportTempPath,
    forProjects: timingHelper.projects.whose({
      productivityRating: {_greaterThan: 0}
    })
  });

  let reportFile = readFile(exportTempPath);
  let report = JSON.parse(reportFile);

  appLog(`${report.length} entries exported (${exportTempPath}).`);
  appLog('');

  return report;
}

function processReport(report, isDraft) {
  // Load projects and find related tasks
  appLog('Parsing time report...');

  // Find related tasks
  var projectsByDay = {};

  report.forEach((entry) => {
    // Ignore invalid entries
    if (!entry.day) {
      return;
    }

    // Map to known projects (and combine unknown projects in fallback)
    let project = config.projects.find(
      (project) => project.name == entry.project
    );
    if (!project || project.inactive) {
      project = config.projects.find((project) => project.name == '[FALLBACK]');
      if (!project) {
        return;
      }
    }

    // Initialize global arrays
    var projectsForDay = projectsByDay[entry.day] || {};
    var projectEntry = projectsForDay[project.name] || {
      name: project.name,
      code: project.code,
      task: project.task,
      time: 0,
      notes: []
    };

    // Add duration
    projectEntry.time += entry.duration;

    // Add notes/title
    var sanitizedTitle = (entry.activityTitle || '')
      .replace(/\* /g, '')
      .replace(/ \| \d+ new items?/g, '')
      .replace(/\(Entries shorter.*/g, '')
      .replace(/\(Untitled.*/g, '')
      .replace(/(Slack \|[^|]*)\|.*/g, '$1');
    if (
      sanitizedTitle.length > 0 &&
      (entry.activityType != 'App Usage' || entry.duration > 1 / 60)
    ) {
      projectEntry.notes.push({
        duration: entry.duration,
        description: sanitizedTitle
      });
      projectEntry.notes.sort((n1, n2) => n1.duration < n2.duration);
    }
    let sanitizedNotes = [
      ...new Set(projectEntry.notes.flatMap((note) => note.description))
    ].splice(0, config.options.maxDescriptionTasks);
    projectEntry.title =
      sanitizedNotes.length > 0
        ? sanitizedNotes.join('\n')
        : project.defaultNote;
    projectEntry.title =
      (isDraft ? config.options.draftPrefix + '\n' : '') + projectEntry.title;

    // Combine back to global arrays
    projectsForDay[project.name] = projectEntry;
    projectsByDay[entry.day] = projectsForDay;
  });

  appLog(`Entries found for ${Object.keys(projectsByDay).length} days.`);
  appLog('');

  // Pre-process and preview report
  var totalTime = 0;

  Object.keys(projectsByDay).forEach((day) => {
    appLog(`Processing ${day}...`);
    let projectsForDay = projectsByDay[day];

    // Sum time entries
    var dayTotalTime = 0;

    Object.keys(projectsForDay).forEach((project) => {
      var entry = projectsForDay[project];

      // Remove insubstantial projects
      if (entry.time < config.options.minimumEntryLength) {
        delete projectsForDay[project];
        return;
      }

      // Round time spent
      entry.time = roundUpHours(entry.time);

      dayTotalTime += entry.time;
    });

    totalTime += dayTotalTime;

    // Sort projects
    sortedProjectsForDay = {};
    config.projects.forEach((project) => {
      if (projectsForDay[project.name]) {
        sortedProjectsForDay[project.name] = projectsForDay[project.name];
      }
    });
    projectsByDay[day] = sortedProjectsForDay;

    logCollection(
      sortedProjectsForDay,
      ['time', 'name', 'task', 'title'],
      [roundUpHours(dayTotalTime).toString(), '(Total hours)']
    );
    appLog('');
  });

  appLog(`Total tracked hours: ${roundUpHours(totalTime)}`);
  appLog('');

  return projectsByDay;
}

//////////////////////////////////
//           HARVEST            //
//////////////////////////////////

function submitToHarvest(projectsByDay) {
  if (Object.keys(projectsByDay).length == 0) {
    appLog('No entries to submit.');
    return;
  }

  // Load projects and find related tasks
  appLog('Mapping projects with Harvest...');

  let harvestProjects = harvestRequest(
    'GET',
    '/projects?is_active=true',
    'projects'
  );

  config.projects.forEach((project) => {
    let harvestProject = harvestProjects.find((hp) => hp.code == project.code);

    if (harvestProject) {
      project.projectId = harvestProject.id;
      project.projectName = harvestProject.name;
    } else {
      throw new Error(`Could not find project with code '${project.code}'.`);
    }

    let harvestTasks = harvestRequest(
      'GET',
      `/projects/${project.projectId}/task_assignments`,
      'task_assignments'
    );
    let harvestTask = harvestTasks.find((ht) => ht.task.name == project.task);

    if (harvestTask) {
      project.taskId = harvestTask.task.id;
    } else {
      throw new Error(
        `Could not find task with name '${project.task}' in project '${project.code}'.`
      );
    }

    appLog(`* ${project.name} – ${project.code} (${project.projectId})`);
  });

  appLog('');

  // Push each day
  var results = [];
  Object.keys(projectsByDay).forEach((day) => {
    appLog(`Pushing time entries for ${day}...`);
    let entries = projectsByDay[day];

    // Delete old entries
    let oldEntries = harvestRequest(
      'GET',
      `/time_entries?from=${day}&to=${day}`,
      'time_entries'
    );
    if (oldEntries.length > 0) {
      oldEntries.forEach((oldEntry) => {
        harvestRequest('DELETE', `/time_entries/${oldEntry.id}`);
      });
      appLog(`Deleted ${oldEntries.length} old entries.`);
    }

    // Create new entries
    var totalHours = 0,
      totalEntries = 0;
    Object.values(entries).forEach((entry) => {
      let project = config.projects.find(
        (project) => project.name == entry.name
      );
      if (!project) {
        throw new Error(
          `Project mapping '${entry.name}' failed during submission.`
        );
      }
      harvestRequest('POST', `/time_entries/`, 'time_entries', {
        project_id: project.projectId,
        task_id: project.taskId,
        spent_date: day,
        hours: entry.time,
        notes: entry.title
      });
      totalHours += entry.time;
      totalEntries++;
    });
    appLog(`Submitted ${totalEntries} new entries.`);
    results.push(
      `${day}: Submitted ${roundUpHours(
        totalHours
      )} hours in ${totalEntries} tasks`
    );

    appLog('');
  });

  if (results.length > 0) {
    displayNotification(results.join('\n'));
  }
}

function harvestRequest(method, endpoint, objectName, body, startCollection) {
  let url =
    (endpoint.indexOf('https') == -1 ? 'https://api.harvestapp.com/v2' : '') +
    endpoint;
  var results = startCollection || [];

  var sanitizedBody = JSON.stringify(body || {})
    .replace("'", "\\'")
    .replace('\\', '\\\\');

  var request =
    "curl -X '" +
    method +
    "' '" +
    url +
    "' \
     -H 'Harvest-Account-Id: " +
    config.harvest.account +
    "' \
     -H 'Authorization: Bearer " +
    config.harvest.token +
    "' \
     -H 'User-Agent: " +
    config.harvest.agent +
    "' \
     -H 'Content-Type: application/json' \
     -d $'" +
    sanitizedBody +
    "'";

  var response = JSON.parse(app.doShellScript(request));

  results = results.concat(response[objectName]);

  if (response && response.links && response.links.next) {
    results = harvestRequest(
      method,
      response.links.next,
      objectName,
      body,
      results
    );
  }

  return results;
}

//////////////////////////////////
//           UTILITIES          //
//////////////////////////////////

function readFile(path) {
  const data = $.NSFileManager.defaultManager.contentsAtPath(path);
  const str = $.NSString.alloc.initWithDataEncoding(
    data,
    $.NSUTF8StringEncoding
  );
  return ObjC.unwrap(str);
}

function appLog(msg) {
  console.log(msg);
  app.doShellScript(
    'echo `date +"%Y-%m-%d %H:%M:%S"` \'' +
      msg.toString().replace("'", "''") +
      "' >> '" +
      config.options.logFilePath +
      "'"
  );
}

function displayNotification(msg) {
  app.displayNotification(msg, {
    withTitle: 'Timing + Harvest',
    soundName: 'Frog'
  });
}

function roundUpHours(hours) {
  let roundUp = 1 / config.options.hourRoundUp;
  return Math.ceil(hours * roundUp) / roundUp;
}

function parseDate(str) {
  let components = str.split(/\D+/).map((s) => parseInt(s));
  components[1] = components[1] - 1; // adjust month
  return new Date(...components);
}

function removeTime(date) {
  return new Date(date.getFullYear(), date.getMonth(), date.getDate());
}

function logCollection(collection, columns, footer) {
  let maxWidth = 50;
  let truncate = (str) =>
    str.length <= maxWidth ? str : str.slice(0, maxWidth - 3) + '...';
  let capitalize = (str) => str.charAt(0).toUpperCase() + str.slice(1);

  // Get max length
  var columnWidths = {};
  Object.keys(collection).forEach((key) => {
    var value = collection[key];
    for (var i = 0; i < columns.length; i++) {
      var contentLength = Math.max(columns[i].length, (footer[i] || '').length);
      value[columns[i]]
        .toString()
        .split('\n')
        .forEach((line) => {
          contentLength = Math.max(
            Math.min(maxWidth, line.length),
            contentLength
          );
        });
      columnWidths[columns[i]] = Math.max(
        contentLength,
        columnWidths[columns[i]] || 0
      );
    }
  });

  // Generate header
  var headerText = '';
  columns.forEach((column) => {
    headerText += ` ${capitalize(column)}${' '.repeat(
      columnWidths[column] - column.length
    )} |`;
  });
  appLog(` ${`-`.repeat(headerText.length - 1)} `);
  appLog(`|${headerText}`);
  appLog(`|${`-`.repeat(headerText.length - 1)}|`);

  // Generate blank line
  var blankLine = '';
  columns.forEach((column) => {
    blankLine += ` ${' '.repeat(columnWidths[column])} |`;
  });
  appLog(`|${blankLine}`);

  // Generate content
  Object.keys(collection).forEach((key) => {
    var value = collection[key];

    // Find number of lines
    var numberOfLines = 1;
    columns.forEach((column) => {
      numberOfLines = Math.max(
        numberOfLines,
        value[column].toString().split('\n').length
      );
    });

    // Output lines
    for (var line = 0; line < numberOfLines; line++) {
      var lineOutput = '';
      columns.forEach((column) => {
        let lines = value[column].toString().split('\n');
        let content = line < lines.length ? truncate(lines[line]) : '';
        lineOutput += ` ${content}${' '.repeat(
          columnWidths[column] - content.length
        )} |`;
      });
      appLog(`|${lineOutput}`);
    }

    // Generate blank line
    var blankLine = '';
    columns.forEach((column) => {
      blankLine += ` ${' '.repeat(columnWidths[column])} |`;
    });
    appLog(`|${blankLine}`);
  });

  // Generate footer
  var footerText = '';
  for (var i = 0; i < columns.length; i++) {
    let content = footer[i] || '';
    footerText += ` ${content}${' '.repeat(
      columnWidths[columns[i]] - content.length
    )} |`;
  }
  appLog(`|${`-`.repeat(footerText.length - 1)}|`);
  appLog(`|${footerText}`);
  appLog(` ${`-`.repeat(footerText.length - 1)} `);
}