For important general, and especially safety related information, please see https://catrob.at/webbricks.

These bricks can be extremely dangerous, so read the information provided on this page carefully. Really!!

The "Send web request" and "Get image from" bricks allow you to access the internet in order to get some information from, e.g., wikipedia, including images from the web. Images can be directly used as looks of an object, and this can for instance be used to retrieve parts of real world maps if a web service allows to do so. Please note that many of these services require API keys, which can be either free of charge but require registration, or commercial and cost money. Make sure to read the warnings on https://catrob.at/webbricks for dangers associated with the use of these bricks. 

To take full advantage of the capabilities of the web bricks, you can use the "join" function to build more complex web addresses required for accessing relevant content, as well as the "regular expression" function to extract relevant information from text your project downloaded with the "Send web request" brick.

Also, you will have to explore what the result of the "Send web request" looks like. In case of many web pages, the answer string can be huge, and at the moment cannot yet be easily inspected in Catrobat's apps. It may be easier and better to download the webpage with a normal web browser such as Chrome, and then edit the downloaded html file to inspect its content. However, you can also now use a "Write variable __ to file __" brick (the file is stored in the "Catrobat" folder) and then inspect the content of the written file using an editor such as QuickEdit (Google Play link). QuickEdit is especially useful as it allows to search in the document using regular expressions. 

Note that a lot of textual information on web pages is generated on the client side, i.e., in the browser, in most cases by using javascript code executed on the user's phone. The text is thus visible to a human in a browser window, but is not contained in the text retrieved with the "Send web request" brick. It can also concern images fetched by the "Get image from" brick. We are currently exploring ways how to resolve these issues.

Use the "Compute" button in the formula editor to try out what the result of the pattern extraction computes to. The function can also report helpful error messages in case the regular expression patterns are not correctly formatted.

For a comprehensive list of the capabilities of regular expressions, please consult the official documentation for the Android version of Catrobat's apps: https://developer.android.com/reference/java/util/regex/Pattern

Note that the "Send web request" and the "Get image from" bricks identify themselves with the following user agent to the server:  Mozilla/5.0 (compatible; Catrobatbot/1.0; +https://catrob.at/bot)

If the "Send web request" brick does not work, an error code from https://en.wikipedia.org/wiki/List_of_HTTP_status_codes may be returned as the result in the variable. The "Get image from" brick instead will show a warning. 

The bricks call a server at the address given through the URL in the first parameter. The result can be either:

• The text-string (html, json, xml, …) returned by the server addressed by the URL (success).
• An error code (error), e.g., 503 (for a complete list see, e.g., https://en.wikipedia.org/wiki/List_of_HTTP_status_codes).
• 504, as the error code for a timeout (60 seconds). 504 is the standard gateway timeout error code.
• In case the app has no internet connectivity, either because the internet has been turned off, or because the permission to access the internet has been taken away from the app, you will be prompted to change your device settings or cancel project execution.

The brick waits till the result has been retrieved, an error has been returned, the script it is contained in is stopped or restarted, or a timeout has been reached. 

Note that robots.txt files on the remote server should be honored.

Note that different scripts of the project that run at the same time can contain web bricks, so there can be more than one connection waiting for an answer at the same time.

A maximum of 10 web requests can be open simultaneously at any given time by a project. If the running project attempts to open more than 10 at the same time, instead of opening the connection, the corresponding 11th etc web request brick will immediately deliver the error code 429. This error code corresponds to the situation that there are too many requests (RFC 6585) – the user has sent too many requests in a given amount of time. This is intended to limit the request rate, e.g., in order to make denial of service attacks a little bit more difficult. Of course there are other, obvious ways how to nevertheless conduct denial of service attacks on third party web services. Please absolutely refrain from trying this, i.e., never EVER, I repeat, NEVER send an inappropriately large number of web requests in short time to a specific web server, as this may degrade the service for other users, and thus may lead the service provider to block our user agent from accessing the web service, thus precluding the use of the service for Catrobat users in the future. Note the related warnings on https://catrob.at/webbricks w.r.t. this issue and what to do if you suspect illegal use of Catrobat's software and services. Denial of service attacks are criminal activities that can have far reaching consequences. You have been warned. 

In case the web-Brick returns a huge text that fills up all the memory, the web connection will be cancelled, the answer discarded, and the error code 400 is returned. This is the code for a "bad request": The server cannot or will not process the request due to an apparent client error (e.g., malformed request syntax, size too large, invalid request message framing, or deceptive request routing).

When the project is paused, or a "Start scene ..." or "Continue scene ..." is executed, all web requests that have not yet been answered are immediately cancelled. When the user later resumes the project from the pause menu, or the scene is reentered using a "Continue scene ..." brick, all web requests that were cancelled previously are now executed again, and the bricks in which they are used are continuing to wait for their answers, also resetting the timeout countdown back to the initial value of 60 seconds.

If, in case of the "Send web request" brick, the url points to some resource that is not in text form, e.g., a jpg image, the behavior is not defined and implementation dependent. The returned string can, e.g., be the binary content of the jpg file, or an error, e.g., 400 for “bad request”. 

Note that short urls are correctly retrieving the information from the short urls' target pages. 

If you want to help in the future development of Catrobat, please contact us. Catrobat is a free open source software project run mostly by pro-bono volunteers, and everyone is invited to contribute. For various ways how to contribute, not only by coding, please see https://catrob.at/contributing. 

Examples

Random jokes:

When scene starts
  Send web request to
    'https://official-joke-api.appspot.com/random_joke'
    and store answer in
    "joke"
  Note 'The joke api returns data like the following: {"id":10,"type":"general","setup":"What kind of shoes does a thief wear?","punchline":"Sneakers"}'
  Set variable "joke question" to regular expression( '"setup":"(.*?)"' , "joke" )
  Set variable "punchline" to regular expression( '"punchline":"(.*?)"' , "joke" )
  Speak "joke question" and wait
  Wait 1 second
  Speak join( "punchline" , '!' ) and wait

Geographical querying:

When scene starts
  Send web request to
    join( 'https://geocode.xyz/' , join( latitude , join ( ',' , join( longitude , '?geoit=json' ) ) ) ) )
    and store answer in
    "geographic location"
  Note 'The geocode.xyz service returns data like {   "statename" : {},   "distance" : "0.000",   "elevation" : "20",   "state" : "UK",   "latt" : "51.50354",   "city" : "LONDON",   "prov" : "UK",   "geocode" : "LONDON-MCRXA",   "geonumber" : "3154700960970",   "country" : "United Kingdom",   "stnumber" : "10",   "staddress" : "DOWNING STREET",   "inlatt" : "51.50354",   "alt" : {      "loc" : {         "staddress" : "DOWNING STREET",         "stnumber" : "10",         "postal" : "SW1A 2AA",         "latt" : "51.50354",         "city" : "LONDON",         "prov" : "UK",         "longt" : "-0.12768",         "class" : {}      }   },   "timezone" : "Europe/London",   "region" : "Greater London, England",   "postal" : "SW1A 2AA",   "longt" : "-0.12768",   "remaining_credits" : {},   "confidence" : "1",   "inlongt" : "-0.12768",   "class" : {},   "altgeocode" : "SYNERGY-MCRXA"}'
  Set variable "current city" to regular expression( '"city" : "(.*?)"' , "geographic location" )
  Set variable "current region" to regular expression( '"region" : "(.*?)"' , "geographic location" )
  Set variable "local country code" to regular expression( '"state" : "(.*?)"' , "geographic location" )
  Send web request to
    join( 'https://fourtonfish.com/hellosalut/?cc=' , "local country code" )
    and store answer in
    "hello in json format"
  Set variable "hello in the local language" to regular expression( '"hello":"(.*?)"' , "hello in json format" ) )
  Speak join( "hello in the local language" , join( '. You are now in ' , join( "current city" , join( ' in ' , "current region" ) ) ) )