PicoCalc WebMite MMBasic - help needed retrieving catfact.ninja API data

Hi Community,

I’ve been trying to retreive and display text data (on my PicoCalc with a Pico 2W fitted) obtained from the simple catfact.ninja API (that doesn’t need a key) based on the WebMite User Manual (https://geoffg.net/Downloads/picomite/Archive/WebMite_User_Manual.pdf) - Implementing a TCP Client example on page 30.

My PicoCalc successfully connects to Wi-Fi on boot, and my code is as follows:


Const url$ = “catfact.ninja”
Const query$ = “GET /fact”
Dim buff%(4096/8)
WEB open tcp client url$, 80
WEB tcp client request query$, buff%()
WEB close tcp client
print Json$(buff%(), “fact”)

Unfortunately when run, I get an error on line 5 of this code ‘Error : No response from server’.

This suggests to me that the remote server is recognised, but not it’s responding to my request!

I’ve checked the API’s web site (which is live and definitely works) on my pc for the correct ‘GET’ extension (i.e. ‘/fact’) and also tried entering full web addresses to see if this makes any difference, but no joy.

Can anyone help with what I’m missing?

Many thanks in advance.

I’m not a MMBasic expert.

The catfact.ninja API uses HTTPS not HTTP. From what I can see in the MMBasic manual, MMBasic does not support HTTPS.

The server does not respond to HTTP (port 80) requests.

You make a very interesting point. My rational was that because the WebMite manual I previously referred to uses a HTTPS API (https://openweathermap.org) as a TCP Client example, I therefore assumed that they would have tested the example code before publishing it!

Has anyone out there written any WebMite MMBasic code that successfully retreives HTTPS API data, or is the WebMite manual’s HTTPS example inherently/fundamentally flawed?

Isn’t that because of missing headers? A minimal valid HTTP/1.1 request needs to look like this:

  GET /fact HTTP/1.1\r\nHost: catfact.ninja\r\nConnection: close\r\n\r\n                                                                                                                            

So this should work

Const url$ = "catfact.ninja"                                                                                                                                                                      
Const query$ = "GET /fact HTTP/1.1" + Chr$(13) + Chr$(10) + "Host: catfact.ninja" + Chr$(13) + Chr$(10) + "Connection: close" + Chr$(13) + Chr$(10) + Chr$(13) + Chr$(10)                         
Dim buff%(4096/8)                                                                                                                                                                                 
WEB open tcp client url$, 80
WEB tcp client request query$, buff%()                                                                                                                                                            
WEB close tcp client                                                                                                                                                                            
Print Json$(buff%(), "fact")             

Wow - that’s a header-and-a-half!

Many thanks, it looks like the server is now responding, but the code now hangs on the last line with ‘Error : Invalid JSON data’!

If you enter ‘catfact.ninja/fact’ into the address field of any pc browser you can see the string data I’m after. I thought the JSON function “fact” attribute should in theory work in my code, and the buffer size should be more than enough.

Is there more to printing the JSON string data contained in buff%(), or have I missed something else?

buff%() probably containt raw response with headers so you need to find the double \r\n\r\n that separates headers from body and parse only what comes after it.

Const url$ = "catfact.ninja"                         
Const query$ = "GET /fact HTTP/1.1" + Chr$(13) + Chr$(10) + "Host: catfact.ninja" + Chr$(13) + Chr$(10) + "Connection: close" + Chr$(13) + Chr$(10) + Chr$(13) + Chr$(10)
Dim buff%(4096/8)                                                                                                                                                                                 
                                                                                                                                                                                                  
WEB open tcp client url$, 80                                                                                                                                                                      
WEB tcp client request query$, buff%()                                                                                                                                                            
WEB close tcp client                                                                                                                                                                              
  
'convert raw buffer                                                                                                                                                              
Dim response$ = buff%()
                                                                                                                                                                                                  
'find http headers end (separated by a blank line \r\n\r\n)
Dim bodyStart = InStr(response$, Chr$(13) + Chr$(10) + Chr$(13) + Chr$(10))                                                                                                                       

' extract only json body by skipping the \r\n\r\n separator                                                                                                                  
Dim body$ = Mid$(response$, bodyStart + 4)
                                                                                                                                                                                                  
'-parse only JSON body
Print Json$(body$, "fact")           
1 Like

Many thanks, your responses are teaching me a lot about MMBasic string manipulation when I’ve been so used to only using python.

I’ve updated my script, but I’m now getting: [7] Dim response$ = buff%() Error : Dimensions!

Please can you help me with this?

It was originally planned to support TLS(the transport layer for HTTPS) in MMBASIC, but the author eventually abandoned this as unworkable(having seen how the network support was kludged in I’m not surprised). The author has since stated his preference to use a coprocessor with onboard TLS support for future MMBASIC network-capable devices, so possibly the manual reference originated from something which was never released or will exist in the future.

Basic compiler MachiKania type P for PicoCalc does support HTTPS, which for me is a quite a distinctive point in favour of it, since most of the interpreted languages available for PicoCalc either have no network support or no TLS, which in practice, since most websites will force HTTPS these days, mostly limits you to connecting to your own specifially set up local server.