What is the best way to manage WDL and json files?

Help
1

Hi,
I went through the tutorial video on working with a WDL. I wonder if people use command line in the terminal to move files, editing them using vi in the terminal, and validating WDL and uploading the final version of files on Dockerstore using command line script.

For example, suppose I have a vcf file of sequence data. I need to use a software package, shapeit, to impute missing data in the vcf file. I have a working Linux script.
What I need to do is the following:

  1. Getting a WDL template and make changes to the WDL template to create wgs_shapeit.wdl file.
  2. Find a docker that holds the shapeit software.
  3. upload my wgs_shapeit.wdl to Terra and test it.

Here comes my questions:

  1. Where I should keep my personal copy of wdl file? On github? On Dockerstore? On Terra broad method repository? My goal is to store in a place where I can easily access it and update it. It seem that I can directly make change to my WDL if I upload my WDL to the Terra broad method repository.
  2. To configure my WDL file, I need to upload a json file.
  3. If I start a computing environment, I can access the terminal window of the VM. Are my WDL and json file stored somewhere on my VM directory?
  4. Bottom line is: I don’t want to use GUI interface to manage my wdl and json files. I would like the process to be scriptable.
  5. What is a good editor of WDL and JSON file?

Thanks.

2

Hi @truckload,

Thank you for your questions. Here are some comments/answers:

  1. We recommend using GitHub since it is a great place to organize your files as in addition to the standard version tracking, collaboration, and command line features. There is also a “Dockstore GitHub app” that can sync any updates from your repo. Here is some useful documentation:
    - Getting Started with WDL — Dockstore documentation
    - Dockstore GitHub App — Dockstore documentation
    - Register a Workflow on Dockstore — Dockstore documentation
  2. You can use Dockstore_CLI, which is a command-line tool that can be used to run workflows locally: Launching workflows using WES and the Dockstore CLI — Dockstore documentation
  3. VS Code might be a good option since it has a WDL plug-in available to use: WDL DevTools - Visual Studio Marketplace

Javi

3

Thank you for the clear instruction.

Let me make sure I understand it correctly. The WDL is pulled from dockerstore or any link provided when it is called for analysis on Terra… NOT the version saved on workspace “persistent disk” or bucket.

I can test run WDL “locally” on the VM in terra (terminal window). But I have to upload the final version of WDL to dockerstore, so the correct version is pulled for the “real” analysis.

The problem for me is that I don’t write code or run analysis “locally” on pc. I have access to Linux cluster. All my file editing is done in rstudio, Jupyter Notebook, emacs or vi. I am trying to find ways to get everything done in Terra + github.