In praise of curl

An essential element in the user experience of the new FileMaker App deskspaceCMS is the use of curl to automate the upload of web pages and other resources to your web server. Nick Lightbody explains exactly how curl can be called from a FileMaker script under both OS X and Windows to achieve this.

“curl is used in command lines or scripts to transfer data. It is also used in cars, television sets, routers, printers, audio equipment, mobile phones, tablets, settop boxes, media players and is the internet transfer backbone for thousands of software applications affecting billions of humans daily.” The curl official site

curl — which originally meant “see url” — is a powerful, flexible and highly useful command line tool which enables your FileMaker App to interact with an other computer in order to transfer data.

curl is a brilliant method of enabling your FileMaker App, on OS X and Windows, to interact with other Apps and services by way of file transfer. It is powerful and of industrial quality, so once you figure out how to use it — it just works.

The problem

In our case the problem we had to solve, in order to make large numbers of deskspaceCMS deployments scaleable with a very simple user experience, was how to:

1. upload new or revised web pages to the web server as part of a scripted process;
2. upload images and other resources to the web server as part of a scripted process and how to
3. automatically create unique new directories/folders on the web server to enable each deskspaceCMS user to have their own web site in their own directory without any manual intervention.

The obvious route into making a curl call was through Applescript on OS X and using a Windows Event on Windows OS.

curl can run many different protocols, I have only used the ftp (file transfer) protocol.

See the curl manual here.

The solution

The steps to success where as follows:

1. Syntax: the correct syntax is not complex but contatenating the correct string from Filemaker can be a little tricky.
2. Encoding the URL: unless you encode the url into HTTP compliant characters your call will fail whenever you use any characters outside the HTTP permitted character set.
3. Debugging: providing the user with a preformed command line to use in debugging their connection was highly desirable since the OS X Terminal App or the Windows OS equivalent then take that command line copied and pasted by the user and automatically create a report showing where the call to curl failed.

OS X: to call curl with Perform Applescript []

PART 1: THE FIRST CALL to CURL

We need to use an expression to assemble the correct string for the call to curl, in this case we are setting a variable $curl which is what we then pass as a parameter to Applescript, hence the syntax has to suit Applescript, as well as curl.

NB: It is essential to only use plain single and double quotes, avoid curly quotes, they will stop your ftp call to curl running correctly.

LINE 1: Set variable [ $curl;
//FUNCTIONS TO TRANSFER FILE
Let([
// define our variables
 ftpserver = <ftp server name or IP Address>;
 destfolder = <the target folder below the ftp root folder if any>;
 u = GetAsURLEncoded(<ftp username>);
 pw = GetAsURLEncoded(<ftp password>);
 file1 = $filename1; // local name
 file2 = $filename2; // name on ftp server
 tp = Get(TemporaryPath);
// concatenate the Applescript string — the curl flags go straight after the initial “curl” declaration
// curl flags can be short or long
// -T or — upload-file | to Transfer a file
// --ftp-create-dirs   | When an FTP or SFTP URL/operation uses a path that doesn't currently exist on the server, the standard behavior of curl is to fail.
//                       Using this option, curl will instead attempt to create missing directories.
x = "do shell script \"curl -T '" &
Case( not IsEmpty( tp )
    ;
     tp ) & file1 & "' ftp://" & u & ":" & pw & "@" & ftpserver & "/" & Case( not IsEmpty(destfolder)
    ;
     destfolder & "/"
    ) &
file2 & "\"" ;
// Finally any required syntactic corrections to the string — here we needed to add “private” to the string
p1=Position(x;"/";1;1);
p2=Position(x;"/";1;2);
y=Middle(x;p1+1;p2-p1-1);  
z=Substitute(x;y;"private")
];z)] 
// end of Set Variable [$curl]
LINE 2: Perform Applescript [ $curl ]
LINE 3: Set Variable [ $error; Get(LastError)]

PART 2: THE SECOND CALL TO CURL

If the first call fails, because the required folder does not exist then you can make a second very similar call — just using a different switch after “curl” which will try to create the directory structure that is missing.

"do shell script \"curl --ftp-create-dirs

In my experience this always works and it is a very wonderful thing.

PART 3: THE THIRD CALL TO CURL

Your third call just repeats the first call, which should now work as the directories it is expecting now exist.

PART 4: ERROR HANDLING

Assuming you have used an if[$error<>0] to call the error handling — we now need to create the debug string required under OS X in Terminal to tell the user why their call to curl failed.

LINE 1: Set field [ Field 1
;
Let([x=$curl;
y=Substitute(x;”\””;”|”);
r=”curl — trace -n -v -ssl -l “ & xExtractBetweenDelimiters ( y ; “|” )];r)
] // end of Set Field
LINE 2: Show Custom Dialogue [
//Title
“A problem has occurred with FM Error: “ & $error
;
// Content
“Sorry, but your upload to your web server was unsuccessful.¶¶
You should check your Settings¶¶
The Apple scripted Curl command used was:¶¶
“ & $applescript & “and you can use the string copied into the field below, in OS X Terminal to debug your Web Server settings.¶¶
The current page has been exported to your local folder specified in settings or if none then to your desktop.¶¶
This process will now halt.”
;
// Input field in dialogue shows Field 1 with label
“On OS X — copy this string into Terminal to debug the Curl Command.”
] // end of Show Custom Dialogue
Note: We used this custom function to extract part of the $curl string:
xExtractBetweenDelimiters (text; delimiters)
Let([s=Substitute(text;”\””;”\\” & “\””);
x=Evaluate(Quote( s ));
p1=Position(x;delimiter;1;1)+1;
p2=Position(x;delimiter;1;2);
L=p2-p1;
y=Middle(x;p1;L);
z=Substitute(y;”\\”;””)];z)

WINDOWS OS: to call curl with a Windows Send Event []

This is very similar to the Applescript version but with some small differences in syntax.

PART 1: THE CALL to curl

LINE 1: Set variable [ $curl;
//FUNCTIONS TO TRANSFER FILE (WINDOWS)
Let([
[as previously]
x = "curl -T '" &
Case( not IsEmpty( tp )
    ;
     tp ) & file1 & "' ftp://" & u & ":" & pw & "@" & ftpserver & "/" & Case( not IsEmpty(destfolder)
    ;
     destfolder & "/"
    ) &
file2;
// Finally any required syntactic corrections to the string — here we needed to remove single quotes
y=Substitute(x;"'";"")
];y)
// end of Set Variable [$curl]
LINE 2: Send Event [ “aevt” ; “odoc” ; “cmd /c “ & Quote($curl)]

Interestingly, the calls to curl from FileMaker appear to run much faster on Windows OS than they do on OS X.

Cheers, Nick

Sussex, UK

September 24th, 2016