Userguides - How To

How to change the data type of your image to Uint8 bit in QGIS

1. Check the data type of your image. Right-click on the imagery layer - i - Information from provider. The Data type must be Byte. Follow the next steps if the Data type is Int16, Float 32 etc.

2. Right click on your image Layer Properties - Symbology tab, then customize the display of your image. Select desired channels for Band rendering, adjust brightness and contrast.

   

3. Right-click on the layer’s name, go to the Export in the context menu, Save as…

   

4. Check Output mode as Rendered Image

   

5. Save your image - navigate to the desired folder, input the file name then click OK

6. Use the new image layer as Imagery source when using the Mapflow plugin for QGIS

How to process your own UAV imagery with Mapflow

Unmanned aerial vehicles – UAVs or, more commonly, drones – have become a deeply integrated part of the geomatic industry over the last ten years. This is owing to their increasing usability, falling hardware costs, and easing government regulations. Yet, as more data is available with UAV surveys, more data need to be processed operatively. To process your UAV data you might be looking for some cloud or desktop software to create a mosaic or orthophoto. Do you know that you can easily publish your data with Openaerialmap and analyze (say detect and calculate some objects and calculate their areas) with Mapflow QGIS or Mapflow Web?

Let’s take the “UAV buildings” 🏠 Buildings (Aerial imagery) model that extracts the detailed building outlines (the recommended image resolution is 10 cm).

Processing with Mapflow Web

1. Select raster source – you can either use Custom URL (see below how to publish your image with Openaerialmap and get the TMS link) or upload your image as GeoTIFF.

Warning

Currently, a preview of the uploaded image is not possible after loading the image, you will see only the area of its extent.

2. Define the processing Area. The processing area (AOI) must be located within the area of the image extent, otherwise, the area will be cut off by the extent boundaries. The processing area size is calculated by the intersection of the image extent and the AOI.

Important

Image upload requirements: The file size must be less than 512 mb. Both sides image dimmesions must not exceed 30.000x30.000

The image must be georeferenced and the CRS must be one of: * WGS84 (EPSG: 4326) * Web mercator (EPSG: 3857) * UTM (any zone)

If your image doesn’t meet the parameters, we suggest using Mapflow API / QGIS plugin which has more capabilities. Mapflow supports RGB imagery and also processes single-band (panchromatic) imagery, but the AI models are not tuned for such kind of data, so the quality of the result may be worse than expected.

Processing with Mapflow – QGIS

In case you are already used to working with QGIS you need to install the Mapflow plugin. You can upload your own GeoTIFF (up to 1 GB, max. 30000x30000 px.). All raster layers currently loaded in your QGIS (1) are visible in the drop-down list (2) and can be selected for upload. Open files in the Additional options (3) also adds your item to the tree of QGIS layers.

Important

Please, consider the requirements specified on the page with Models reference when uploading your own images for AI-mapping processing. Contact us if you have a large dataset of images or your file size exceeds our limits.

Use Openaerialmap as an imagery publication tool

OpenAeriaMap is an open collection of UAV imagery data, crowdsourced by users. The project is supported by a consortium of companies developing open source software and services for working with geospatial data. As soon as your aerial image is published on Openaerialmap it’s displayed on the public map and can be streamed using TMS/WMTS protocols or downloaded as GeoTIFF file. Both ways are OK to work with Mapflow.

1. Copy link to TMS and paste it into the “Custom imagery URL” in your new Mapflow processing.

2. Check if you see the image on the map, go through the next steps (AI model, processing params) to and start the processing.

Preparing and optimizing the large size images

Here are few tips on how to prepare and optimize your data and reduce the image size to upload it faster and not to exceed the Mapflow upload limit.

Usually UAV image is an RGB compiosite provided as GeoTIFF of 16 or 8 bit. The type must be Byte (8 bit). If the Data type is Int16 or Float32 etc, please follow the instruction Userguides - How To. Alternatively: use the preprocessing script for preparing your image for Mapflow processing.

You can reduce the size of the image using GDAL translate. (https://gdal.org/) E.g. using JPEG compression. gdal_translate -co compress=JPEG input.tif output.tif By default the compression quality is 75% (gdal_translate -co compress=JPEG -co jpeg_quality=75 input.tif output.tif) but it doesn’t really impact the quality of the Mapflow mask whenever the resolution of the input iage meets the recommended params.

The same can be done using QGIS interface:

Tell us if you have more tips to share with the community or if you have more questions – we are ready to help.

Run the flow!

How to view results using Kepler.gl

Kepler.gl is an open source tool designed for geospatial data analysis. It is a simple yet powerful for displaying and exploring geodatasets.

To view the processing results in the Mapflow, select the required processing and press the button “Open in kepler.gl”.

Note

You can share your processing view in Kepler by copying the open URL (right click on “Open in kepler.gl” –> Copy Link Address)

Using the Kepler you can change the visual properties of data, set filters, and choose a background map.

Layers tab

Click on the layer name to bring up the Layer settings from the drop-down menu. To hide all data, click on the eye icon.

These settings allow you to choose a more suitable type of received data:

• Fill color. You can choose any color from the palette for polygons, and also hide the display of data by changing the position of the slider. You can change the transparency of polygons (property Opacity) in the additional settings of this function.

• Stroke color. You can choose any color from the palette for outlining polygons, as well as completely remove the stroke. You can change the transparency of the stroke (property Opacity) In the additional settings of this function.

• Stroke width. Controls the thickness of the stroke.

• Height. Allows you to view data with heights in 3D format. Set the desired coefficient and select the attribute of the layer with heights.

Filters tab

This tab allows you to add a filter of interest by a specific attribute of the layer (as in this case, the filter is set by classes with different typology of buildings).

Interaction tab

You can select or remove attributes that will be visible in the menu that appears when you hover over an object. It is also possible to turn on the panel indicating longitude and latitude.

Base map tab

Here you can choose the styles of the map, as well as choose to display its various layers.

Iterative mapping tutorial

This workflow is proposed for the AI assisted mappinf of the crop fields usinf Sentine-2 images. If you use Mapflow for agricultural landuse management, we suggest you have a look at the methodology described below. It can help you streamline the mapping process so you could focus on the end goal and results instead. All you need is our QGIS plugin (a.k.a. Maflow for QGIS) installed. If you haven’t connected the Sentinel-2 data yet, contact us.

Before we begin, however: Mapflow will digitize the fields for you, but the imagery for it to work on you need to select yourself. We strongly recommend you preview images before processing them because their quality may vary. Ones with high cloud cover (including cloud shadows) or no-data pixels in the AOI will likely produce poor results. Here are examples of two images that will inevitably yield unsatisfactory results:

Fields detected using a clouded Sentinel-2 image

An example of a Sentinel-2 image that has no-data pixels for the AOI.

Sometimes you may not be able to find a single suitable image for your entire AOI. In that case, we recommend you process the best available images separately and then combine the results. On average, you will need to process ten images for a territory of 1000 km².

What is an Iterative mapping process

1. Start processing on the Sentinel-2 images:

   • In QGIS, load your AOI as a layer;

   • Open the plugin and go to Providers tab; select Sentinel-2 in the drop-down list in the upper left corner and your AOI layer as the Area below;

   • Adjust the search criteria: select the search period in From and To; adjust the maximum allowed cloud cover and the minimum required intersection percentage in Additional Filters;

   • Click Search imagery; it may take up to a minute, depending on the size of your AOI and the search period;

   • The search results will show up in the table below. Each row describes a single image. You can sort the table by date and cloud cover and you can further filter the results by adjusting the dates and the filters as you did in the previous step.

   • Search results have also been added to your QGIS project as a layer called “Sentinel-2 metadata”. The layer is synchronized with the table: every time you select a row in the table, the corresponding image is selected in the layer, and vice versa.

   • Now, select an image and double click on its table row. You will see it got added to the project as a raster layer. AOI have a real value (they’re not black or transparent).

Note

This is a preview, - a coarse representation of the actual image. The image itself will have a 30-times better resolution (10m instead of 300m in the preview). You’ll want an image which in which cloud don’t cover your AOI and all pixels within your

• Once you’ve made your choice, make sure the image row is selected in the table, go to Processing tab; type in a name for your processing, select “Sentinel-2 Fields” in the Mapflow Model drop-down list and check that your AOI layer is selected as the Area and the Imagery source is set to Sentinel-2; if everything looks good, hit Start processing.

Repeat the same for five to ten images covering the same area so the best available image is processed for every part of your AOI.

2. Next, create a new polygon layer called “results” into which we’ll merge the results.

3. Download the processing results by double-clicking the corresponding rows in the Processings table. Group raster and vector layer for each processing for simplicity. Create a mutually exclusive group for these groups.

4. Reorder the layers so that the best results are on top. Normally, it takes about ten seconds to determine if the results are good enough.

5. Choose the topmost result layer pair (vector and raster). Select all objects in the vector layer and copy them into the “results” layer.

6. In the “results” layer, select and delete poorly digitized polygons. Now, the “results” layer has gaps which should be filled with successfully contoured polygons from the other result layers.

7. Select the next best processing and put its vector layer on top of the “results” layer. Select well-contoured polygons from this layer and copy them into the “results” layer.

If there are still gaps in the “results” layer, repeat step #7 until you the all gaps are filled in. After several iterations, you should get fully contoured agricultural fields for your AOI.

Note

Watch this video for a visual example of this iterative markup process.

How to run bulk processing using Mapflow API

In case you have multiple polygons to process or update, it can be boring to upload them one by one using Web or GIS user tools. In this case, you’d better think of using Mapflow Mapflow processing API. In this example let’s assume we have a list of polygons indicating the populated places borders and we want to extract features like “buildings” with Mapflow processing API.

1. To make it more realistic let’s download some populated places borders for the sample area in Uzbekistan using Openstreetmap. To do this we can make a query with Overpass Turbo API like this: We can use QuickOSM plugin in QGIS which is very friendly when it comes to downloading a managable volume of data from Openstreetmap.

Authorization with Mapflow token

Let’s Mapflow token for further authorization.

import base64

token = input("token:") # Your Mapflow token goes here. Obtain it at https://app.mapflow.ai/account/api

username, password = base64.b64decode(token).decode().split(':')
print(username, password)

Now you get your token decoded as a username and a password and use it for the authorization with the requests.

Create the project (it’s optional)

Hint

It might be useful to organise your processing with the projects. To do this create the new project with the following API method.

import requests
import json

url = "https://api.mapflow.ai/rest/projects"
headers = {
    'Content-Type': 'application/json'
    }

payload = json.dumps({
        "name": "My new project"  # Your project name
    })

response = requests.request("POST", url, headers=headers, auth=(username,password), data=payload)

if  response.status_code == 200:
    print(response.text)
else:
    print(f"Request failed")
    print(response.text)

Here we get the response containing the project ID, that we can use to create the processings in this specific project.

Response example:

    {
    "id": "fb49b97e-51ec-4b31-872f-d1411284de85",
    "name": "My new project",
    ...
}

See more in Get default project

Prepare AOIs for the processings

Let’s save areas of interest with the properties as a GeoJSON file as it’s simple and straightforward format to be used in any application or GIS software. Then we open this file and create a python dictionary to loop through all GeoJSON features that we are going to use as AOI geometries for creating the processing. Like this:

with open('<path to the file>', 'r') as file:  # Define your GeoJSON file path
geojson_data = json.load(file)

for feature in geojson_data['features']:
    name = feature['properties']['name']   # Extract the "name" property from OSM data
    print(name);

Let’s check if we created the data array from our file and display all the features by their names. At the next step, we will use the name property to define the processing. The “name” is optional yet it’s more convenient to work with the results afterwards.

Run the Processings

Now we are ready to create the processing for each AOI using its geometry.

url = "https://api.mapflow.ai/rest/processings"

for feature in geojson_data['features']:
    name = feature['properties']['name']
    geometry = feature['geometry']
    payload = json.dumps({
        "name": name,
        "projectId": "fb49b97e-51ec-4b31-872f-d1411284de85",  # Here is your project Id to link the processing to the specific project.
        "wdName": "🏠 Buildings",
        "geometry": geometry
    })
    response = requests.request("POST", url, headers=headers, auth=(username,password), data=payload)

    if  response.status_code == 200:
        print(f"Request successful: {name}")
    else:
        print(f"Request failed for feature: {name}")
        print(response.text)

If everything was done correctly - the list of successfully created processing will be displayed.

Download all the results using Mapflow API

When all processings are complete you can download easily the results for each one.

If you have one processing with the multiple AOIs (by default the number of AOIs in one processing is limited to 10) you can run a single API call to download the results:

curl --location 'whitemaps-internal.mapflow.ai/rest/processings/<ID>/result' \
--header 'Authorization: Basic <YOUR API TOKEN>' -O <YOUR PATH TO FILE>.geojson

In case of the multiple processings, you might find it useful to run the small script.

1. Get the list of all “ids” and “names” by processing:

import requests
import json

url = "https://api.mapflow.ai/rest/projects/fb49b97e-51ec-4b31-872f-d1411284de85/processings"

response = requests.request("GET", url, auth=(username,password))

json_data = response.json()

values = []
for item in json_data:
    if "id" in item and "name" in item:
        values.append((item["id"], item["name"]))

if  response.status_code == 200:
    for id, name in values:
        print(f"{id}, {name}")
else:
    print(response.text)

Response example:

5f748822-c94a-4233-ae1e-7622973bf9b5, Бой
87f258e6-c1ce-4deb-8155-ccd6b21ac237, Авазли
1e24c0e5-9b6f-4a26-9970-82dfb1f67807, Avazali
0316606a-4c04-4196-bdab-6495af4bb7b0, Авлиятепа
38435f06-6b85-420e-86f8-3ebef4755480, Акбуйра
33ffec93-ef2b-4ead-a51e-30cad1bcb71d, Аксулат
c8f73cf5-8a44-4c36-8b2e-49b27f7f5da5, Актепа
6eaca828-41ec-4ce5-aea4-b9f039211bbe, Арабхана
...

2. Save results for all the listed processings:

output_json = '<path to the folder>' # Define local folder to save files

for id, name in values:
    response = requests.request("GET", url + id + '/result', headers=headers)

    if  response.status_code == 200:
        with open(output_json_file + name + '.geojson', 'w') as geojson:
            geojson.write(response.text)
            print(f"File saved")
    else:
        print(f"Request failed")
        print(response.text)