Tutorial: HD LIDAR with QGIS 3.32
In this article you’ll find tools for LIDAR data: LASTools, WhiteboxTools, OpenLIDAR Tools. Step-by-step installation, licensing and configuration. You’ll also find a presentation of CloudCompare.
As we saw in the article LIDAR data in QGis 3.32, the new version of QGis integrates a series of processes for point cloud data. This is only the first step. As time goes by, new processing functions will be added to make QGis a powerful tool for processing LiDAR data. For the time being, however, it is necessary to use other processing providers to perform certain operations. Here we look at the other tools available in, or alongside, QGis.
2- Download LIDAR HD data from IGN and load it into QGis
3- Tools for LIDAR data in QGis 3.32
Tools natively available in QGis 3.32 for LIDAR data
A new native processing provider has been integrated for point cloud algorithms: the provider currently supports the following algorithms:
- Information: provides basic point cloud metadata (number of points, extent, crs, etc.).
- Convert format: converts the point cloud to a different format, e.g. from las to laz.
- Reproject: reproject the point cloud to a different CRS.
- Fix projection: fix (assign) the CRS of a point cloud file.
- Clip: clip a point cloud by cutting out one or more polygons
- Merge: merge several point clouds into a single file
- Tile: create tiles from input data
- Thin: create a thinner version of the point cloud
- Boundary: export a vector layer containing the point cloud boundaries
- Density: exports a raster file in which each cell contains the number of points in that cell’s area.
- Export to raster: exports point cloud data to a 2D raster grid
- Export to vector: export point cloud data to a vector layer with 3D points
- Export to raster (TIN): export data from a point cloud to a 2D raster grid using point triangulation.
- Filter: extract a subset of the point cloud using PDAL expressions
This set of tools enables you to perform the most common operations on point clouds.
For more specific processing, there are three other tools that can be integrated into the QGis processing toolbox:
- LASTools
- WhiteboxTools
- Open LIDAR Tools
There is also a complementary open source product to QGis, CloudCompare.
Before looking at the tools, we need to make one important point. At the time of writing, these tools have been interfaced with previous versions of QGis. The notion of a point cloud layer did not yet exist. When you open a QGis point cloud editor, you’ll see a settings window with the words Source layer. The drop-down menu will display the loaded layers, allowing you to define them as the processing input.
When you open the additional toolbox processing window, you’ll see a window like this:
Even if you have loaded a layer into QGis, processing will have to point to the original files.
Similarly, the files resulting from these processes will be files on disk and will not be loaded automatically at the end of the process, even if this option appears checked in the processing window.
Needless to say, under these conditions, if you opt for the default output option, a temporary file, you won’t get any results at all!
LAStools, LIDAR data processing
It’s presented as a double toolbox, with a (small) free part and a (large) paid part requiring a license. The license, incidentally, is only valid for one year, after which compulsory maintenance is required to continue using the product.
RapidLasso offers three types of LAStools license:
- Personal license: this license is intended for individuals who wish to use LAStools for non-commercial purposes.
- Commercial license: for companies wishing to use LAStools for commercial purposes.
- Trial license: this license allows you to use LAStools free of charge for 30 days.
I’ll leave it to you to find out more on their website: https://rapidlasso.de. But, as I had a lot of trouble understanding what you can and can’t do with the version that’s installed by default, here’s the summary that will save you a LOT of time.
LASTOOL, license or no license?
The only explanation you’ll find for using LASTOOLS is the following paragraph::
There are TWO parts to LAStools. One part is OPEN source.
The other part is CLOSED source and requires licensing for most commercial or government use. If you’re unsure please email ‘info@rapidlasso.de’ before using LAStools.
For education and evaluation purposes you can use and test LAStools for free. Some free version tools are may limited in the maximum count of lidar points in one single file to avoid commercial usage.
Let’s put this text through the decoder and see things for what they are:
1- There’s only one version of the toolbox, and that’s the one you’ll be installing (see next paragraph).
2- All treatments work as soon as you have installed and configured the treatment provider.
3- “Licensed” treatments work with or without a license. However, depending on the processing, from 1.5 million points upwards if you don’t have a license, the processing will add either noise or black bands to the resulting file. In short, output quality will be “slightly” degraded.
4- If you purchase a license, you will receive a key that will enable you to remove this point limit.
5- If you look hard enough, you’ll find a forum in which the designer of LASTOOLS indicates a way of getting around this problem of point limits. He simply advises you to use a LASTOOLS processing that allows you to divide a LIDAR file into tiles of less than 1.5 million points, apply the desired processing (classification, ground detection, etc.) to the tiles, then reconstruct a single output from the tile outputs.
6- Don’t feel guilty if you opt for this system, because what you need to know is that if you’ve paid for a license and your LIDAR file is over 15 million points long (IGN files are on average 22 million…) you’ll get a memory allocation error message that will block processing. Once you’ve consulted Rapidlasso as indicated in the error message, you’ll be told to… break it down into tiles of less than 15 million points, apply the desired processing (classification, ground detection, etc.) to the tiles, then reconstruct a single output from the tile outputs!
Installing LASTOOLS for QGis
Here’s how to install LASTOOLS on QGis, because it’s not all plain sailing either:
Install the LAStools plugin using the extension manager:
For LAStools, you need to install not only the plugin but also the executables:
Download the LAStools software from here and unzip it to a permanent location, e.g. “c:/LAStools”
! !! Important: Make sure the path does not contain spaces or special characters.
In Qgis, go to the Preferences / Options menu.
Navigate to the Processing tab on the left, then double-click on Processing Providers, then double-click on LAStools.
Navigate to the LAStools folder you downloaded in step 1, e.g. “C:/LAStools”.
Click any other box, e.g. “Scripts”, then click OK.
A small detail to remember: if you run one of the processes and you get this error message:
There’s no need to look for the source of this UTF-8, it’s just a message to let you know that you haven’t set the processing provider options to the right LASTOOLS directory.
LASTOOLS tool structure
The LASTOOLS toolbox consists of three types of tools:
- Basic tools that take a file as processing input. The code for these tools can be found in the LAStools/bin directory you specify in the Processing Provider Options.
- Similar tools, but with a directory as processing input. All LAS or LAZ files in this directory are processed. The processing name ends with Pro, and the code for these processes is loaded by the plugin. It is therefore not located in the same directory as the previous ones, but in C:\Users\username\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\LAStools\LAStoolsProduction
- Pipelines, i.e. tools which automatically chain together a series of previous tools. They can be found in the C:\Users\username\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\LAStools\LAStoolsPipelines directory.
This information will be useful if you want to create your own processing workflows. We’ll look at this in detail in the LAStools chapter later in the tutorial.
WhiteboxTools for LIDAR data
Here’s another toolbox with two parts, a (large) free-to-use part and a (small) part under annual or perpetual license.
Many of the processes in the free part are not yet included in the default QGis processes. It is therefore useful to use this toolbox to complete your point cloud processing possibilities.
The following is a list of available treatments. Treatments that are crossed out require a license.
For installation, you’re beginning to see that it’s a habit, so you need to take a few precautions:
For WhiteboxTools, you need to install not only the plugin, but also the binaries/executables for the operating system concerned:
1-Install the plugin using the Qgis extension manager:
2-Download the binaries for your operating system here.
Unzip the downloaded folder and place it in a safe place, e.g. C:/whitebox/
! !! Important: Make sure the path does not contain spaces or special characters.
3-In Qgis, go to Preferences/Options.
Navigate to the Processing tab on the left, then double-click on Processing Suppliers, then double-click on WhiteboxTools.
Navigate to the executable file “whitebox_tools” (“whitebox_tools.exe” under Windows) in the folder you downloaded in step 2.
Click any other box, e.g. “Scripts”, then click OK.
If you don’t have a license and try to run one of the licensed processes (it’s up to you to find out which ones they are, as there’s nothing to distinguish them from the others) you’ll get an error message:
The Open LIDAR Toolbox plug-in
Open LiDAR Toolbox is a QGIS plug-in for one-step processing of airborne LiDAR data, from point cloud to LiDAR visualizations. The required input is an unclassified point cloud in LAZ/LAS format, and the tool returns the results needed for interpretative mapping of archaeological features. In addition, several other tools are available for multi-stage data processing. The tools are optimized for archaeology, but have wider application for anyone interested in the visual inspection of airborne LiDAR, for example, topographic mapping.
Open LiDAR Toolbox is a “wrapper” that uses several excellent tools under the hood: GDAL, GRASS, native QGIS tools, LAStools, Whitebox Tools and RVT. Consequently, the following plug-ins must be installed before Open LiDAR Toolbox: LAStools, Whitebox Tools and RVT. If these plug-ins are not already installed, they will be automatically installed with Open LiDAR Toolbox. However, the additional steps (installation of executables for LAStools and WhiteboxTools) must be followed.
To install the plug-in, go to the extension manager.
If you haven’t installed the two previous plug-ins, LASTools and WhiteboxTools, download and install the executables as described above.
ONE tool for LIDAR data processing
This is an algorithm pipeline that takes an airborne LiDAR point cloud and produces all the derivatives essential for archaeology and anyone interested in the visual analysis of LiDAR data or using it for topographic mapping. The pipeline introduces several additional steps compared with a traditional approach. The results are a moderate improvement in ground point classification (ASPRS class 2) and a significant improvement in building classification (ASPRS class 6). The latter is particularly important for digital terrain models (DTMs) and digital feature models (DFMs).
As these tools were developed for archaeological purposes, they have been optimized to distinguish between three types of terrain model:
This figure shows the three types of model:
DTM – digital terrain model
DSM – digital surface model
DFM – Digital Feature Model.
This plugin is particularly useful when calculating this third type of model.
CloudCompare software
CloudCompare is free, open source software for processing, analyzing and visualizing 3D data. It is used by a wide community of professionals, including geomaticians, engineers, architects and scientists.
CloudCompare supports a wide variety of 3D data formats, including point clouds, voxel clouds, surfaces and polygonal models. It enables a variety of operations to be performed on 3D data, including :
- Visualization
- Filtering
- Re-ranking
- Segmentation
- Reconstruction
- Merge
- Measurements
- Export
CloudCompare is a powerful and versatile tool that can be used for a variety of 3D data processing, analysis and visualization tasks. It’s free and open source, making it accessible to a wide audience.
Here are just a few examples of what you can do with CloudCompare:
- Visualize point clouds and 3D surfaces
- Filter point clouds to remove noise and artifacts
- Reclassify point clouds to make them easier to visualize
- Segment point clouds to identify distinct objects
- Reconstruct surfaces from point clouds
- Merge point clouds to create larger 3D models
- Measure distances, surfaces and volumes
- Export 3D data in different formats
If you’re working with 3D data, CloudCompare is an indispensable tool alongside QGis. It’s free, open source and easy to use.
Above all, it will be a great help in fine-tuning your processing. Indeed, the tools we have seen above are processing tools: you define the input data and obtain the output data. With CloudCompare, you work interactively. You can test different parameter settings in a more user-friendly and efficient way.
CONCLUSION
Whatever the type of project, with QGis you already have the tools you need to achieve your goals. The new version 3.32 is just the beginning of a more complete and rational integration of all these tools. In future versions, tools will be progressively integrated into the QGis core, plug-in designers will support the new “point cloud” layer type and redundant tools will gradually disappear.
