OpenCV - Lane detetion, both yellow and white lane

I have road photos from the racing game.

I want to detect yellow and white lanes.

If I use RGB space,

def select_rgb_white_yellow(image): 
    # white color mask
    lower = np.uint8([123, 116, 116])
    upper = np.uint8([186, 172, 160])
    white_mask = cv2.inRange(image, lower, upper)
    # yellow color mask
    lower = np.uint8([134, 121, 100])
    upper = np.uint8([206, 155, 87])
    yellow_mask = cv2.inRange(image, lower, upper)
    # combine the mask
    mask = cv2.bitwise_or(white_mask, yellow_mask)
    masked = cv2.bitwise_and(image, image, mask = mask)
    return masked

I only get white lanes.
So tweaked little bit, using HLS space, by using this code.

def convert_hls(image):
    return cv2.cvtColor(image, cv2.COLOR_RGB2HLS)

Then, extracting the yellow and white lane again,

def select_white_yellow(image):
    converted = convert_hls(image)
    # white color mask
    lower = np.uint8([0, 0, 0])
    upper = np.uint8([0, 0, 255])
    white_mask = cv2.inRange(converted, lower, upper)
    # yellow color mask
    lower = np.uint8([ 10,   0, 100])
    upper = np.uint8([ 40, 255, 255])
    yellow_mask = cv2.inRange(converted, lower, upper)
    # combine the mask
    mask = cv2.bitwise_or(white_mask, yellow_mask)
    return cv2.bitwise_and(image, image, mask = mask)

Then it cannot detect white lane anymore. Is there good way to detect both white and yellow lane?

Here are all RGB colour code that I found

Yellow
c2974A (194, 149, 74)
a07444 (160, 116, 68)
b38e55 (179, 142, 85)
867964 (134, 121, 100)
ce9b57 (206, 155, 87)
ce9853 (206, 152, 83)

white
b4a59d (180, 165, 157)
b9a99a (185, 169, 154)
baaca0 (186, 172, 160)
867e79 (134, 126, 121)
7b7474 (123, 116, 116)
827d7c (130, 125, 124)

To extend the comment: this is an implementation of a 2 stage approach. Take some time to look at the intermediate images/masks to fully understand everything that happens.

Cropping to region of interest
You can automate this, but I cheated a little and did it manually. The cropped sky area will rarely ever have road surface, this is an easy solution that suffices (for now) I think. Similarly, I also cut of the HUD boxes on the right hand side, as they have similar gray colors as the road and were interfering. It's more neat to draw black boxes over them, so they are excluded from processing.

Isolating road
Convert the cropped image to HSV an select only gray-ish values. After some noise removal I improved the mask using findContours to draw the convexhull. If performance is an issue, you could maybe skip it by tweaking the close mask step.

Selecting lines
Using the mask you can create an image of only the road surface. You can use this for color separation without having to worry about also selecting surroundings. My result is not perfect, but I assume you have larger versions of the images which will give better results.

Result:

Code:

import cv2
import numpy as np 

# load image
image = cv2.imread('pw12b.jpg')
# crop image
h,w = image.shape[:2]
image = image[200:h-20,20:550]
# create hsv
hsv = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
# set lower and upper color limits
low_val = (0,0,0)
high_val = (179,45,96)
# Threshold the HSV image 
mask = cv2.inRange(hsv, low_val,high_val)
# remove noise
mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel=np.ones((8,8),dtype=np.uint8))
# close mask
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel=np.ones((20,20),dtype=np.uint8))

# improve mask by drawing the convexhull 
ret, contours, hierarchy = cv2.findContours(mask, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
for cnt in contours:
    hull = cv2.convexHull(cnt)
    cv2.drawContours(mask,[hull],0,(255), -1)
# erode mask a bit to migitate mask bleed of convexhull
mask = cv2.morphologyEx(mask, cv2.MORPH_ERODE, kernel=np.ones((5,5),dtype=np.uint8))

# remove this line, used to show intermediate result of masked road
road = cv2.bitwise_and(image, image,mask=mask)

# apply mask to hsv image
road_hsv = cv2.bitwise_and(hsv, hsv,mask=mask)
# set lower and upper color limits
low_val = (0,0,102)
high_val = (179,255,255)
# Threshold the HSV image 
mask2 = cv2.inRange(road_hsv, low_val,high_val)
# apply mask to original image
result = cv2.bitwise_and(image, image,mask=mask2)

#show image
cv2.imshow("Result", result)
cv2.imshow("Road", road)
cv2.imshow("Mask", mask)
cv2.imshow("Image", image)

cv2.waitKey(0)
cv2.destroyAllWindows()