After analyzing different designs, the simplest model to follow and that works correctly is the following.

First, it is thresholded again with HSV, but the values ​​of the first algorithm have changed and they were used for the web. Now the color filter works in a more robust way according to the following values:

(im_HSV[:, :, 0] <= 10) & (im_HSV[:, :, 1] >= 100) & (im_HSV[:, :, 2] > 30)

It has been adopted as the center, the center of the image. However, this may vary depending on the initialization of Gazebo, so it should be impeded that the center is in the center of the first threshold with the first initialization. For ease, the center of the image has been simply taken (in columns).

After that, we have searched among the neighboring lines of row 250. Taking this reference, we start from a fixed value that will vary depending on how far it is from the center. With this we obtain the reference to which we will denote as «ref». It will serve as the braking force at the fixed speed that is applied.

The turn is dependent on speed and reference «ref». Therefore, at higher speed, there will be more amount of turn and vice versa. This is important since it is not the same to rotate at high speed than at low speed, since it would derail quickly.

This post has been updated with a video showing how it works. We see that it completes the lap with a simulation time of about 26 seconds.