Skip to contents

Convert or copy originals into the project's data folder

Source: R/convert_data.R
convert_data.Rd

The project publishes what is in data/: the analysis reads it, the metadata describes it and the submission compendium carries it. This brings an original into that folder in a format that will still open in twenty years. It never writes anywhere else and never touches the original.

Usage

convert_data(path, to = NULL, overwrite = FALSE, also = character(0))

Arguments

path

File or folder to read, as a single string: relative to the working directory – the project root, when you have opened the project's .Rproj – or absolute. A folder is read whole, subfolders included. The originals are only ever read.

to

Where the results go. Left out, the data/ folder of the project set_paper() named; with no project named, it has to be given. There is no default folder: easypaper never takes the working directory for it.

overwrite

FALSE (the default) writes only what is missing from to or older than its original; TRUE rewrites the lot.

also

Extensions to treat as already open for this call, with or without the dot, e.g. "las". For good: list them under open-formats: in the easypaper: block of the project's _quarto.yml.

Value

The paths written, invisibly.

Details

Each file takes one of three roads, decided by its extension alone:

  • Converted to .csv: .xlsx and .xls, one .csv per sheet (needs readxl); .sav, .dta and .sas7bdat, one .csv per file (needs haven). These are closed formats with an open equivalent faithful enough to publish. A file that cannot be read – corrupt, or not really what its extension says – is named in a warning and skipped, and the rest of the batch goes on. A missing readxl or haven is reported once per call, with the files it held back.

  • Copied byte for byte, because they already are the open format and converting one would destroy it rather than open it:

    • text and tables: .csv, .tsv, .txt, .json, .geojson, .xml, .yml, .yaml

    • containers: .parquet, .nc, .h5, .hdf5, .sqlite, .db, .gpkg

    • spatial: .shp with its sidecars (.shx, .dbf, .prj, .cpg, .sbn, .sbx, .qix), .kml, .gml, .tif, .tiff, .asc

    • sequences and trees: .fasta, .fa, .fastq, .fq, .nwk, .tre

  • Named on screen and left where it is: anything else. A proprietary instrument file, an ArcGIS project, a photograph of a field notebook – nothing here can tell whether it belongs in the paper, or what "the table" inside it would even be.

When your field uses an open format this list has not heard of, pass its extension in also for a single call, or list it under open-formats: in the easypaper: block of the project's _quarto.yml, and every call copies it from then on.

A format conversion, not a transformation: filtering, recoding and cleaning belong in the analysis chunk that needs them, where a reader can check them. Every conversion costs something – an .xlsx loses its formulas, an .sav its value labels – which is why the original is never moved or altered.

Subfolders of a source folder are read but not reproduced: everything lands flat in data/. When two originals want the same name there, one keeps it – whichever comes first in alphabetical order by path – and the rest are refused with a warning saying which was kept and which was not, rather than one table quietly overwriting another. Names that differ only in case count as the same name, because the deposit has to unpack on a file system that cannot tell Counts.csv from counts.csv.

Examples

src <- file.path(tempdir(), "originals")
dir.create(src, showWarnings = FALSE)
write.csv(head(iris), file.path(src, "iris.csv"), row.names = FALSE)
out <- file.path(tempdir(), "data")

convert_data(src, to = out)   # copied: a .csv is already open
#> data/ updated: 1 file(s) -- iris.csv

# A workbook: one .csv per sheet.
convert_data(readxl::readxl_example("datasets.xlsx"), to = out)
#> data/ updated: 3 file(s) -- datasets_mtcars.csv, datasets_chickwts.csv, datasets_quakes.csv
list.files(out)
#> [1] "datasets_chickwts.csv" "datasets_mtcars.csv"   "datasets_quakes.csv"  
#> [4] "iris.csv"             

# An extension it does not know, copied as it is when named in `also`.
writeLines("x y z", file.path(src, "points.las"))
convert_data(file.path(src, "points.las"), to = out, also = "las")
#> data/ updated: 1 file(s) -- points.las
list.files(out)
#> [1] "datasets_chickwts.csv" "datasets_mtcars.csv"   "datasets_quakes.csv"  
#> [4] "iris.csv"              "points.las"           
unlink(c(src, out), recursive = TRUE)