Python: How to create and visualize point clouds

1. Introduction

Point cloud applications are everywhere: robots, self-driving cars, assistance systems, healthcare, etc. Point cloud is a 3D representation suitable for processing real-world data, especially when the geometry of the scene/object is required, such as the distance, shape and size of the object.

A point cloud is a set of points that represent a scene in the real world or an object in space. It is a discrete representation of geometric objects and scenes. In other words, a point cloud PCD is a collection of n points, where each point Pi is represented by its 3D coordinates:

Note that it is also Add some other features to describe the point cloud, such as RGB color, normals, etc. For example, RGB colors can be added to provide color information.

2. Point cloud generation

Point clouds are usually generated using 3D scanners (laser scanners, time-of-flight scanners and structured light scanners) or computer-aided design (CAD) models. In this tutorial, we will first create and visualize a random point cloud. We will then use the Open3D library to sample points from the 3D surface to generate it from the 3D model. Finally, we'll see how to create them from RGB-D data.

Let’s start by importing the Python library:

import numpy as np
import matplotlib.pyplot as plt
import open3d as o3d

2.1 Random Point Cloud

The easiest way is to randomly create a point cloud. Note that we generally do not create random points to process except when creating noise for a GAN (Generative Adversarial Network).

Usually, point clouds are represented by (n×3) arrays, where n is the number of points. Let's create a point cloud with 5 random points:

number_points = 5
pcd = np.random.rand(number_points, 3)# uniform distribution over [0, 1)
print(pcd)

We could print the points directly, but it's not very efficient, especially in most applications if the number of points is large. A better approach is to display them in 3D space. Let’s visualize it using the Matplotlib library:

# Create Figure:
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.scatter3D(pcd[:, 0], pcd[:, 1], pcd[:, 2])
# label the axes
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_zlabel("Z")
ax.set_title("Random Point Cloud")
# display:
plt.show()

Random Point Cloud Visualization

2.2 Sampling Point Cloud

Required for direct processing of 3D models time. Therefore, sampling point clouds from their three-dimensional surfaces is a potential solution. Let's start by importing the bunny model from the Open3D dataset:

bunny = o3d.data.BunnyMesh()
mesh = o3d.io.read_triangle_mesh(bunny.path)

Or import it as follows:

mesh = o3d.io.read_triangle_mesh("data/bunny.ply")

Next, display the 3D model to see how it looks. You can move your mouse to view from different viewpoints.

# Visualize:
mesh.compute_vertex_normals() # compute normals for vertices or faces
o3d.visualization.draw_geometries([mesh])

Rabbit 3D Model

To sample a point cloud, there are several methods. In this example, we uniformly sample 1000 points from the imported mesh and visualize them:

# Sample 1000 points:
pcd = mesh.sample_points_uniformly(number_of_points=1000)


# visualize:
o3d.visualization.draw_geometries([pcd])

Rabbit Point Cloud

We can save the created point cloud in .ply format as follows:

# Save into ply file:
o3d.io.write_point_cloud("output/bunny_pcd.ply", pcd)

2.3 Point cloud from RGB-D data

RGB-D data is generated using RGB -D sensor (such as Microsoft Kinect) collected, which provides both RGB images and depth images. RGB-D sensors are widely used in indoor navigation, obstacle avoidance and other fields. Since RGB images provide pixel colors, each pixel of the depth image represents its distance from the camera.

Open3D provides a set of functions for RGB-D image processing. To create a point cloud from RGB-D data using Open3D functions, simply import two images, create an RGB-D image object, and finally calculate the point cloud as follows:

# read the color and the depth image:
color_raw = o3d.io.read_image("../data/rgb.jpg")
depth_raw = o3d.io.read_image("../data/depth.png")


# create an rgbd image object:
rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(
color_raw, depth_raw, convert_rgb_to_intensity=False)
# use the rgbd image to create point cloud:
pcd = o3d.geometry.PointCloud.create_from_rgbd_image(
rgbd_image,
o3d.camera.PinholeCameraIntrinsic(
o3d.camera.PinholeCameraIntrinsicParameters.PrimeSenseDefault))


# visualize:
o3d.visualization.draw_geometries([pcd])

Colored point clouds generated from RGB-D images

3, Open3D and NumPy

Sometimes you need to switch between Open3D and NumPy. For example, let's say we want to convert a NumPy point cloud to an Open3D.PointCloud object for visualization, and use Matplotlib to visualize a 3D model of a rabbit.

3.1 From NumPy to Open3D

In this example, we create 2000 random points using the NumPy.random.rand() function, which starts from the uniform distribution of [0,1] Create a random sample. We then create an Open3D.PointCloud object and set its Open3D.PointCloud.points feature to random points using the Open3D.utility.Vector3dVector() function.

# Create numpy pointcloud:
number_points = 2000
pcd_np = np.random.rand(number_points, 3)


# Convert to Open3D.PointCLoud:
pcd_o3d = o3d.geometry.PointCloud()# create point cloud object
pcd_o3d.points = o3d.utility.Vector3dVector(pcd_np)# set pcd_np as the point cloud points


# Visualize:
o3d.visualization.draw_geometries([pcd_o3d])

##Open3D Visualization of Random Point Cloud

3.2 从 Open3D到NumPy

这里，我们首先使用Open3D.io.read_point_cloud()函数从.ply文件中读取点云，该函数返回一个Open3D.PointCloud对象。现在我们只需要使用NumPy.asarray()函数将表示点的Open3D.PointCloud.points特征转换为NumPy数组。最后，我们像上面那样显示获得的数组。

# Read the bunny point cloud file:
pcd_o3d = o3d.io.read_point_cloud("../data/bunny_pcd.ply")


# Convert the open3d object to numpy:
pcd_np = np.asarray(pcd_o3d.points)


# Display using matplotlib:
fig, ax = plt.subplots(subplot_kw={"projection": "3d"})
ax.scatter3D(pcd_np[:, 0], pcd_np[:, 2], pcd_np[:, 1])
# label the axes
ax.set_xlabel("X")
ax.set_ylabel("Y")
ax.set_zlabel("Z")
ax.set_title("Bunny Point Cloud")
# display:
plt.show()

使用 Matplotlib 显示的兔子点云

4、最后

在本教程中，我们学习了如何创建和可视化点云。在接下来的教程中，我们将学习如何处理它们。

The above is the detailed content of Python: How to create and visualize point clouds. For more information, please follow other related articles on the PHP Chinese website!