Can I run terminal commands such as SFTP/SSH through a mathematica notebook?

Question

I'm generating an array using Mathematica which produces a single .dat file. I then use SFTP/SCP to transfer it to the staff Linux cluster.

This isn't a particularly difficult task but is there a way to execute the terminal command through a mathematica notebook as if it were the MacOS terminal?

Answer 1

You can use CopyFile with a RemoteFile parameter, including the necessary Authentication options

CopyFile[
    "/tmp/yourlocalfile.dat",
    RemoteFile[
        "sftp://host.example.com/tmp/yourremotefile.dat",
        Authentication -> <|
            "Username" -> "user",
            "SSHKey" -> File["/path/.ssh/id_rsa"]
            |>
    ]
]

CopyFile it's very flexible and accepts also ExternalStorageObject and CloudObject.

There are public test servers to play with "SCP"/"SFTP", this one is read-only, you can CopyFile from there to your local drive, for tests.

RemoteFile[
        "sftp://test.rebex.net/pub/example/readme.txt",
        Authentication -> <|
            "Username" -> "demo",
            "Password" -> "password"
            |>
    ]

Run and RunProcess, ExternalEvaluate["Shell", "Cmd"] or even RemoteRun could be used too, if you already have a script that does the file transfer.

RunProcess[{"ls", "--help"}]