Meta Project

The Unofficial Meta Documentation

Meta CGI programming

CGI programming for real

Programming for the Common Gateway Interface (not the computer generated image stuff here).

Why CGI programming? It is old, nobody does it anymore, it is difficult error prone and a million more reasons can be found on the internet.

Yes, and everything that is on the internet is true, off course!

"Better alternatives are available". Perl? ASP? Also old. PHP? Seriously? Have you ever debugged a script that does not return anything visible in your browser?

Let's start by saying these alternatives have been around but they do have their own troubles.

First of all you will see that CGI programming in C is not advised in a lot of places. All of these are not coming from C programmers for sure, but from advocates of the other alternatives. For sure the main disadvantage from programming in C is that it can be pretty tricky to get everything right. This is valid certainly when code involves the use of pointers and the programmer will have to know about his business.

On the other hand a lot of CGI programming, for example in Perl is depending on running an interpreter that will interpret you CGI code, or a compiler that will compile it first. Thus making it a load on the server.

A pre-compiled script, like done with C does not have this disadvantage.

If you use Meta to create your pre-compiled CGI scripts, you will have the best of both worlds.

Meta is very suitable for a task like CGI scripting. Meta programs are very readable for us as humans and that means errors are more likely to be found before the program is released on the web. Meta is a SAVE language. And because Meta scripts result in binaries for your system no compiler or interpreter is needed to run on the server, it is fast and has a low footprint. Hence Meta combined with CGI scripting will make a realistic alternative for other options you might have, especially if you really love to program instead of copy and paste options and settings for framework generated templates.

Did you know that you can use CGI programming for your database connection as well? Because Meta can connect to your database it is also an alternative in practical situations. At this stage in development of Meta the way Meta currently handles connections with databases is still the only way to handle connections to a MySQL or MariaDB database. As time will pass other options will become available.

You will learn that CGI programming is not so bad after all if you use Meta! Everything you really need is available.

So I experimented with creating some CGI scripts.

The start

As always the start is the most basic step. Just try to get some output from your CGI script first!

The very first a script should produce is the header information. If any output is done ahead of that time you will be out of luck. So almost always start with printing that lines:


; Write header information
write/line {Content-type: text/html
}

This will print the famous line "Content-type: text/html\n\n" for you.

Compiling and testing

A simple script can be tested in an early stage in the webconsole. But as soon as you need real input data you must move over to compiling your script. To some degree this also can be tested locally but the proof of the pudding is in the eating, so over to the hosting solution. Make sure you are allowed to run CGI scripts there.

Then make sure your newly compiled program.com is given proper rights.


chmod 755 program.com

And copy it to the right location using your FTP program or the terminal you use to connect to the hosting.

"Happy testing!"

Using Windows as development for Linux server

When you are using a Windows machine for your development while your server for your website is some Linux machine then you might feel you have a problem. And you are right. Your program will initially run as a Windows program and not a Linux program. You can upload your program.com file as your project.cgi executable but that will not work. If you have terminal access on the Linux machine, you can upload your source and start your compilation from there using the ./run method. Problem solved.

But if you do not have this? There is another workaround for this. I created a cgi program that will run your cgi script first, making the APE do its work and adapt your script for the Linux environment.

If you would like to get this progam, you can contact me for it. If a lot of requests come in, I can put this up through a link.

Environment variables

Getting info from environment variables


Meta [
    Title:   "Show environment variables using Meta CGI"
    File: environmentcgi.meta
    Author:  ["Arnold van Hofwegen"]
    Rights:  "Copyright (c) 2023-2023 Arnold van Hofwegen"
    License: {
        PD/CC0
        http://creativecommons.org/publicdomain/zero/1.0/
    }
    Purpose: {
        Show environment variables for Meta CGI.
    }
]

; Write header information
write/line {Content-type: text/html
}

text: select/case system/environment "DOCUMENT_ROOT"
write/line "<p>"
write "DOCUMENT_ROOT = "
write/line text
write/line "</p>"

bye

This will give the info from the desired variable. Other Environment variables to try to get info from are:


GATEWAY_INTERFACE
HTTPS
HTTP_ACCEPT
HTTP_ACCEPT_ENCODING
HTTP_ACCEPT_LANGUAGE
HTTP_REFERER
HTTP_COOKIE
HTTP_CONNECTION
HTTP_HOST
HTTP_SEC_FETCH_DEST
HTTP_SEC_FETCH_MODE
HTTP_SEC_FETCH_SITE
HTTP_SEC_FETCH_USER
HTTP_UPGRADE_INSECURE_REQUESTS
HTTP_USER_AGENT
HTTP_X_ACCEL_INTERNAL
HTTP_X_REAL_IP
PATH
QUERY_STRING
CONTENT_LENGTH
REMOTE_ADDR
REMOTE_PORT
REQUEST_METHOD
REQUEST_URI
SCRIPT_FILENAME
SCRIPT_NAME
SCRIPT_URI
SCRIPT_URL
SERVER_ADDR
SERVER_ADMIN
SERVER_NAME
SERVER_PORT
SERVER_PROTOCOL
SERVER_SOFTWARE
UNIQUE_ID

To get right to the relevant part of the QUERY_STRING you can use the extended form of getting the variable


text: find/tail select/case system/environment "QUERY_STRING"  "="

Referer will be empty when the completed script will be called directly.

So we create the form to run the script using POST

502 Bad Gateway

What now?

POST troubleshooting

When you first try to change a CGI that uses GET and make it into a CGI that uses POST, chances are you will run into trouble.


"when I try to POST"
(104)Connection reset by peer: ap_content_length_filter: apr_bucket_read() failed

The solution to this problem lies in the way to reproduce this error, and it is the thing that resulted in your 502 Bad Gateway error page.


All it takes to produce the above error with Apache is to send a post to a cgi script that does not handle the post data.

So the suggestion is that you make sure the POST data gets handled. If you do not use the data, then just read it in from stdin and throw it out in the garbage. Or close the stdin handle and httpd will handle it from there for you.

Cryptic right? Right. It just means we need to


; Read from STDIN, this is for POST
data-in: read/line

in our script. You can see this in action elsewhere on this page. When you now compile your script it will run and expect you to enter something on the commandline and press enter.

This fixes the problem.

And the HTTP_REFERER, if you display or want to use it will be filled with usefull information.

Securing your script

According to sources your scripts can be executed by invoking them directly from the browser address bar, as soon as the location is known. Making use of the information available you could insert code that quits the script as soon as it is not a POST, or if its HTTP_REFERER is not on your site or any allowed site for that matter.

More on securing your scripts later.

Creating a page with a generated svg image

This task is a two part effort. One part is creating the page that calls for the image. The other part is the program that generates the image as an svg. The svg does not need to be saved on the server in the first place, it can be added to the webpage as generated.


<html><head></head>
<body><div>This script gets the result of the simplesvg cgi program that is in the cgi-bin<br>
    <IMG SRC="../../cgi-bin/simplesvg" alt="This should be a generated svg file"></div>
</body>
</html>

Making the simplesvg

Creating an svg is relative easy. The svg documentation gives the specification of various elements that can be used to construct the svg. Also using a program like Inkscape that can save or export files in svg format can help. The only thing you yourself will have to do is make some minor modifications to the generated file and you're in business.

The one important thing to note is the header information that is different for a svg file!


Meta [
    Title:   "Simple CGI image"
    Author:  ["Arnold van Hofwegen"]
    Rights:  "Copyright (c) 2023-2023 Arnold van Hofwegen"
    License: {
        PD/CC0
        http://creativecommons.org/publicdomain/zero/1.0/
    }
    Purpose: {
        Test cgi from cgi-bin locations and forms
    }
]
write/line {Content-Type: image/svg+xml
}
svg-part-1: to string! {<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->

<svg
   width="111.93318mm"
   height="24.509819mm"
   viewBox="0 0 111.93318 24.509819"
   version="1.1"
   id="svg106383"
   inkscape:version="1.1.2 (0a00cf5339, 2022-02-04)"
   sodipodi:docname="simple-meta2.svg"
   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
   xmlns:xlink="http://www.w3.org/1999/xlink"
   xmlns="http://www.w3.org/2000/svg"
   xmlns:svg="http://www.w3.org/2000/svg">
  <sodipodi:namedview
     id="namedview106385"
     pagecolor="#ffffff"
     bordercolor="#666666"
     borderopacity="1.0"
     inkscape:pageshadow="2"
     inkscape:pageopacity="0.0"
     inkscape:pagecheckerboard="0"
     inkscape:document-units="mm"
     showgrid="false"
     inkscape:zoom="2.0192011"
     inkscape:cx="133.221"
     inkscape:cy="112.91594"
     inkscape:window-width="1646"
     inkscape:window-height="991"
     inkscape:window-x="26"
     inkscape:window-y="23"
     inkscape:window-maximized="0"
     inkscape:current-layer="layer1"
     fit-margin-top="0"
     fit-margin-left="0"
     fit-margin-right="0"
     fit-margin-bottom="0" />
  <defs
     id="defs106380">
    <linearGradient
       inkscape:collect="always"
       id="linearGradient106965">
      <stop
         style="stop-color:#556b2f;stop-opacity:1;"
         offset="0"
         id="stop106961" />
      <stop
         style="stop-color:}

original-color: "#e71b62"
random/seed now
random-color: random 255
any [
    if random-color < 25 [chosen-color: "#ff0000"]
    if random-color < 50 [chosen-color: "#00ff00"]
    if random-color < 75 [chosen-color: "#0000ff"]
    if random-color < 100 [chosen-color: "#cc6666"]
    if random-color < 125 [chosen-color: "#cc3300"]
    if random-color < 150 [chosen-color: "#ccff00"]
    if random-color < 175 [chosen-color: "#3300ff"]
    if random-color < 200 [chosen-color: "#ffffcc"]
    if random-color < 225 [chosen-color: "#ffff00"]
    if random-color < 250 [chosen-color: "#ff66ff"]
    (chosen-color: original-color)
]

svg-part-2: to string! {;stop-opacity:0"
         offset="1"
         id="stop106963" />
    </linearGradient>
    <linearGradient
       inkscape:collect="always"
       xlink:href="#linearGradient106965"
       id="linearGradient107127"
       gradientUnits="userSpaceOnUse"
       x1="22.465916"
       y1="32.789104"
       x2="190.17657"
       y2="10.260267" />
  </defs>
  <g
     inkscape:label="Laag 1"
     inkscape:groupmode="layer"
     id="layer1"
     transform="translate(-23.202355,-19.493473)">
    <text
       xml:space="preserve"
       style="font-style:normal;font-weight:normal;font-size:26.2042px;line-height:1.25;font-family:sans-serif;display:inline;opacity:1;fill:url(#linearGradient107127);fill-opacity:1;stroke:none;stroke-width:0.264583"
       x="19.321411"
       y="39.890442"
       id="text93574"
       transform="scale(1.0327803,0.96826015)"><tspan
         sodipodi:role="line"
         id="tspan93572"
         style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:26.2042px;font-family:'Linux Biolinum Keyboard O';-inkscape-font-specification:'Linux Biolinum Keyboard O';fill:url(#linearGradient107127);fill-opacity:
1;stroke-width:0.264583"
         x="19.321411"
         y="39.890442">META</tspan></text>
  </g>
</svg>
}
write/line join/with join/with svg-part-1 chosen-color svg-part-2
bye

Piglatin form

Kaj de Vos (a.o. MetaProject.frl) and Nick Antonaccio (a.o. Rebolforum.com) both cooperated on a CGI script using Meta that used a GET request.

I used this as the base for a CGI script with a form that uses the POST method.


Meta [
    Title:   "Piglatin CGI script with form"
    Author:  ["Arnold van Hofwegen, Kaj de Vos, Nick Antonaccio"]
    Rights:  "Copyright (c) 2023-2023 Arnold van Hofwegen"
    License: {
        PD/CC0
        http://creativecommons.org/publicdomain/zero/1.0/
    }
    Purpose: {
        Create a piglatin FORM for Meta CGI usage.
    }
]

; Write header information
write/line {Content-type: text/html
}

; Read from STDIN, this is for POST
data-in: read/line

MAX-CONTENT-LENGTH= 80
text: select/case system/environment "CONTENT_LENGTH"
data-length: 0
if not is-none text [
    content-length: to integer! text
    if content-length > MAX-CONTENT-LENGTH [
        content-length: MAX-CONTENT-LENGTH
    ]
    data-in: copy cut data-in content-length
    content-in: find/tail data-in "="
]

; Read a QUERY string, this is for GET!!
text: find/tail select/case system/environment "QUERY_STRING"  "="

; Now take GET input or POST input
string-input: ""
unless text [
    if content-in [
        text: content-in
    ]
]

; Prepare string for the output
piglatin: ""

if text [
    ; convert text into piglatin
    while all [text  not is-tail text] [
        plc: 0 pec: 0
        if plus-word: find text "+" [plc: count plus-word]
        if perc-word: find text "%" [pec: count perc-word]
        any [if all [plc = 0 pec = 0][look-for: "+" skip-count: 1]
             if any [pec = 0 plc > pec][look-for: "+" skip-count: 1]
             if any [plc = 0 plc < pec][look-for: "%" skip-count: 3]
        ]
        text: if space: find word: text look-for [
            word: copy cut text (count text) - (count space)
            skip space skip-count
        ]
        unless is-empty word [
            string-input: join/with string-input word
            vowel: none
            character: word

            until any [
                is-tail character
                if find "aeiou"  copy cut character 1 [vowel: character]
            ][
                advance character
            ]
            either is-same word vowel [  ; Word starts with vowel
                piglatin: join/with join/with piglatin word
                "yay"  ; way, hay
            ][
                either vowel [
                    piglatin: join/with join/with piglatin vowel
                    copy cut word (count word) - (count vowel)
                ][
                    piglatin: join/with piglatin word
                ]
                piglatin: join/with piglatin "ay"
            ]
        ]
        piglatin: join/with piglatin " "
        string-input: join/with string-input " "
    ]
]

; Write the form

write {<html>
<head><style>
h1 {color: darkolivegreen;}
input[type=text] {width: 80%;}
input[type=submit] {width: 40%;
text-align: center;
background-color: darkolivegreen;
color: white}
table {width: 60%;}
tr {height: 2em;}
.piglatin {width: 100%; border-width: 1px;
border-style:solid ; border-color: black; height: 1.5em;}
</style></head>
<body><h1>Meta Pig Latin Generator</h1>
<p>
<table><tr><td>
    <FORM ACTION="/cgi-bin/piglatin" METHOD="POST">
    <DIV>Enter some text to convert into Pig Latin:</DIV><br>
    <INPUT NAME="data" SIZE="100" MAXLENGTH="80" VALUE="}
write string-input
write {" PLACEHOLDER="Put your text here"><br><br>
    <INPUT TYPE="SUBMIT" VALUE="CONVERT WITH META"><br>
    </FORM>
</td></tr><tr><td>
<div class="piglatin">}
write PIGLATIN
write/line {</div>
</td></tr>
</table>
</p>
</body>
</html>}

Connecting to a MySQL MariaDB database

Now we are talking!

Because we are compiling our scripts connecting to the database is relative quick, but because we are using a call to the commandline interface each time the overall performance is a little less optimal. Especially if more than one databasequery needs to be performed by our script and each must be done by a separate call. (I have not yet experimented with multiple statements at once)

For most usecases this method is comparable with a php script that needs to connect for each page to the database as well and perform a single action. And it will suffice unless the load on the server will get massive.

How does this work

Hidden file with connection info

Because the file with the connect information contains the real information to connect to my database, I really do not want others to see this data. So I need to access the file preferrably using an absolute path location.

Because Meta cannot directly provide access from a root path, we use a trick of constructing that using the root folder that it does know.


Root-folder= /.
Connect-file= join/with Root-folder "path/to/my/hidden/connect-db-info.txt"

Result-file= ./result.txt

Reading the info from the connection info file

Creating the statement

Calling the database from the script

The data retrieved must be precessed, so this will be stored in a temporary file. Here I use the file result.txt, but obviously if traffic increases, a unique name should be chosen for every instance.


Result-file= ./result.txt

Putting it all together

Here is what the data in the connection information file looks like:


HOST=mysqlserver.hosted.net
PORT=3306
USER=username
PASSWORD=password
DATABASE=databasename

And here is a complete script.


Meta [
    Title:   "Test a MySQL MariaDB database connection with Meta"
    Author:  ["Arnold van Hofwegen"]
    Rights:  "Copyright (c) 2023-2023 Arnold van Hofwegen"
    License: {
        PD/CC0
        http://creativecommons.org/publicdomain/zero/1.0/
    }
    Purpose: {
        Get data from a database connection using cURL
    }
]
Root-folder= /.
Connect-file= join/with Root-folder "path/to/my/hidden/connect-db-info.txt"
Result-file= ./result.txt

; Write header information
write/line {Content-type: text/html
}

; Set MySQL server connection parameters
Either connects= try open Connect-file [
    mark-info: 0
    ; read all connection info
    Until (
        is-tail connects
    ) [
        text: take/line connects
        text: either space: find word: text "=" [
            word: copy cut text (count text) - (count space)
            skip space 1
        ]["UNKNOWN"]
        any [
            if word = "HOST" [
                mark-info: mark-info or 16
                MYSQL-HOST: text
            ]
            if word = "PORT" [
                mark-info: mark-info or 8
                MYSQL-PORT: text
            ]
            if word = "USER" [
                mark-info: mark-info or 4
                MYSQL-USER: text
            ]
            if word = "PASSWORD" [
                mark-info: mark-info or 2
                MYSQL-PASSWORD: text
            ]
            if word = "DATABASE" [
                mark-info: mark-info or 1
                MYSQL-DATABASE: text
            ]
        ]
    ]
    Close connects
    if mark-info != 31 [
        write/line join/with "Missing info in file " Connect-file
        unless mark-info and  1 [write/line "<br>No database information present"]
        unless mark-info and  2 [write/line "<br>No password information present"]
        unless mark-info and  4 [write/line "<br>No user information present"]
        unless mark-info and  8 [write/line "<br>No port information present"]
        unless mark-info and 16 [write/line "<br>No host information present"]
        bye
    ]
][
    write/line join/with "Not able to open and read file " Connect-file
    bye
]

; Define the query to execute
QUERY: "SELECT count FROM visit_counter WHERE name = 'testcount';"

; Execute the query and save the result to a file
basic-statement: join/with join/with join/with join/with join/with
                 join/with join/with join/with join/with join/with
                 {mysql -h "} MYSQL-HOST {" -P "} MYSQL-PORT
                 {" -u "} MYSQL-USER {" -p"} MYSQL-PASSWORD
                 {" -D "} MYSQL-DATABASE {" -e "}

statement: join/with join/with join/with basic-statement QUERY {" > } Result-file

Either query-result= run statement [ ; result not 0 means an error occurred
    error: to error! query-result
    write "(Networking) error: "  write/line error
	write/line "Query not executed"
][ ; Process the result file
    Either results= try open Result-file [
        ; Read the value of the counter, always second and last row
        header: take/line results
        counter: to integer! take/line results
        Close results
    ][
        write/line join/with "Not able to open and read file " Result-file
    ]
]

; Update the counter
increment counter
UPDATE-QUERY: join/with join/with "UPDATE visit_counter SET count = " to string! counter
              " WHERE name = 'testcount';"
statement: join/with join/with basic-statement UPDATE-QUERY {"}

Either update-result= run statement [ ; result not 0 means an error occurred
    error: to error! update-result
    write "(Networking) error: "  write/line error
	write/line "<br>Update Query not executed"
][
    write "<br> Site now visited: " write counter write/line " times<br>"
    write "<br> congratulations!<br>"
]

; Clean up the result file
Either remove-result= run join/with "rm " Result-file [ ; result not 0 means an error occurred
    write/line join/with "Not able to remove file " Result-file
][
    write/line "<br><br>Ready"
]

Note that if I compile this on the host using the ./run script I have there also, the progam will execute immediately. To stop this, you could add an extra file that needs to be present inside the true folder the script needs to be in and that misses in the compilation folder on your host.

XMLHTTPRequest

With the XMLHTTPRequest you can embed data from your CGI script inside your webpage.

The way to do this is create a CGI script that creates the data (obvious) and then make sure you create a (is it Javascript?) function that locates the right element in the DOM and replaces it with the data it retrieves from the CGI script.

The location of the script is up to you to edit as suitable for your case.


<html>
<head>
    <title>XMLHTTPRequest test with Meta CGI</title>
    <script>
    function loadHtmlFromCGI() {
        var xhr = new XMLHttpRequest();
        xhr.onreadystatechange = function() {
        if (xhr.readyState == 4 && xhr.status == 200) {
            var container = document.getElementById("html-container");
            container.innerHTML = xhr.responseText;
        }
        };
        xhr.open("GET", "/cgi-bin/dbvisitcounter", true);
        xhr.send();
    }
    </script>
</head>
<body onload="loadHtmlFromCGI()">
    <div><h1>Here the data from our script</h1></div>
    <div id="html-container">Loading HTML from CGI script...</div>
</body>
</html>