Erik L. Arneson — Writer and Software Developer

Update: Org to DOCX with Citations

2023-06-20T00:00:00+00:00

Last year, I wrote about converting Org to DOCX with pandoc. Well, that particular method has needed some improvements. I needed to also support converting Markdown files, and more vitally, I needed to support the new-ish org-cite citation format.

The first thing I did was update to the latest version of Pandoc. Next, I had to learn how Pandoc’s citations work. Note that you have to enable the citations extension as well.

For citations to work, you need to have a Citation Style Language (CSL) file. Zotero comes with a ton of them, so check your Zotero installation for examples.

In the updated fish shell function below, you will want to update both refdoc and csldoc to point to your reference DOCX file and your CSL file, respectively.

function org2docx --description 'Generate a DOCX file using a custom reference document'
    set -l refdoc "$PATH_TO_REFERENCE_DOCX"
    set -l csldoc "$PATH_TO_CSL"
    set -l fromfmt (string match -r '(?:org|md)$' $argv)
    set -l base (basename -s .$fromfmt $argv)

    echo Generating $base.docx ...

    pandoc --from "$fromfmt"+citations \
        --citeproc --csl $csldoc \
        --reference-doc $refdoc -o $base.docx $argv
end

And there you have it! Now you can convert both Org files and Markdown files to DOCX. And I am sorry that you have to use DOCX!

Writing and Reviewing Jupyter Notebooks

2023-05-18T00:00:00+00:00

A recent project involves delivering a finished product as a collection of Jupyter Notebooks. This process involves using Emacs for writing, Git for version control, and a slightly tricky process for enabling non-Jupyter, non-Emacs users to perform document review.

Writing—just like programming—ideally includes a review process before anything is delivered to the client. Even first drafts need at least two readers before delivery. I’ve previously discussed how I use Org Mode and Pandoc to deliver DOCX files, and DOCX or ODF files unfortunately remain the easiest way to track changes and edits among word processor users.

Since Jupyter Notebooks are basically JSON documents, the best way to keep track of changes and revisions is using some kind of version control. Here is one process using Git and GitHub.

When my notebook files are ready for review, converting them to DOCX files is pretty straightforward.

First, I use the jupyter command line tool to convert to Markdown, like this:
```
jupyter nbconvert --to markdown *.ipynb
```

Next, I use Pandoc to convert to DOCX using a reference link.

for file in *.md
    pandoc --reference-doc $path_to_refdoc -o $file.docx $file
end

Renaming files with Dired

At this point, I needed to rename all of the DOCX files and move them to the proper shared folder, so my reviewer could get to them and know what’s going on. We have a naming format for filenames that helps us track project and versions, so all of the files needed to have at least a v1 in them.

Emacs has a file manager called Dired, which contains powerful features that allow you to modify directory contents just like any other buffer. I now had a bunch of files that ended in .md.docx that needed to instead end in -v1.docx. Here is the process I used to easily rename them.

In Emacs, use M-x dired to open the directory.
Use C-x C-q to run dired-toggle-read-only.
Use M-% to run query-replace, and replace .md.docx with -v1.docx.
Finish “writing” the directory with C-c C-c.

All done! It was nice and simple. The DOCX files were finally properly named and ready for review.

An Org-mode to DOCX Pipeline

2022-10-26T00:00:00+00:00

Freelance writers need to deliver documents in the format requested by clients. However, frequently the requested format is not the writer’s preferred working format. I like to write in Org Mode, but many clients prefer delivery in Microsoft Word’s DOCX format.

This is how I generate DOCX files for my clients.

What is Org Mode?

Org Mode is an Emacs package for writing and working with Org files. Org files are highly structured plain text files that may appear to be a text outline, but can do so much more. Org Mode is incredibly versatile, and can be used to track projects, manage schedules, write outlines, and even create documents.

Choosing between Org Export and Pandoc

Org Export runs inside Emacs and is capable of converting Org files to a variety of other formats. While it is very powerful, it also has its idiosyncrasies. For instance, when converting Org files to DOCX files, it uses its own style names such as “Org Title” and “Org Heading 1”.

A second option for converting Org files to DOCX files is Pandoc. Pandoc prides itself on being the Swiss army knife of document format conversion. It handles an impressive variety of document formats and handles a dizzying collection of configuration options.

Since the DOCX files that I create need to be shared with other writers, editors, and reviewers, I need to make sure that they are easy to work with. This influenced my decision. Since Pandoc uses more standard style names, I decided to use it for Org conversion.

Setting up Pandoc

To use Pandoc to generate nice looking DOCX files, you will need to configure a template document. The recommended method for doing this is to generate a default template using Pandoc, and then edit it in Word. I used LibreOffice Writer for this, and it worked just fine.

Install the latest Pandoc using these instructions.

Run the following command to generate reference.docx

pandoc -o custom-reference.docx --print-default-data-file reference.docx

Open reference.docx in your word processor and edit the styles so they meet your needs.

Converting from Org to DOCX

One option for running the conversion is to take advantage of the ox-pandoc package for Emacs. If you will always be using the same configuration for your exports, this is a great option.

However, I need to use a number of different configurations for converting documents, so I tend to run Pandoc from the command line. Recent versions of ox-pandoc support passing options via Org headers, but I still haven’t bothered to set that up. It should be very easy to template this using Yasnippet, though.

I use a custom fish shell function that looks like this:

function org2docx --description 'Generate a DOCX file using a custom reference document'
    set -l refdoc "$PATH_TO_REFERENCE_DOCX"
    set -l base (basename -s .org $argv)
    echo Generating $base.docx ...
    pandoc --reference-doc $refdoc -o $base.docx $argv
end

From my fish shell command line, I can then just run org2docx whatever.org to generate whatever.docx.

I have not found a level of automation that makes my converted DOCX files completely perfect, unfortunately. After conversion, I always open the new file in my word processor to make final tweaks and fixes.

Have fun converting files!

The method I’ve outline in this blog post is straightforward and fits my needs. There are definitely improvements to be made, such as using templates to pass the proper options to Pandoc. Switching to ox-pandoc would mean one fewer reason to leave Emacs, after all.

In recent years, more and more clients are asking for files to be delivered via Google Docs. So far, I have yet to find a good conversion pipeline to get Org files into Google Docs easily. My method right now takes too many manual steps. That’s a problem I would love to solve.

Do you have a conversion pipeline for documents that works for you? Leave me a comment and let me know!