journal_list | How to participate | E-utilities
Kim, Rhee, and Jang: Lane Detection for Parking Violation Assessments

### Abstract

In this study, we propose a method to regulate parking violations using computer vision technology. A still color image of the parked vehicle under question is obtained by a camera mounted on enforcement vehicles. The acquired image is preprocessed through a morphological algorithm and binarized. The vehicle’s shadows are detected from the binarized image, and lanes are identified using the information from the yellow parking lines that are drawn on the load. Whether parking is illegal is determined by the conformity of the lanes and the vehicle’s shadow.

### 1. Introduction

Computer vision systems used in various industrial fields have recently accomplished distinct research achievements for autonomous vehicles as a result of their applications in the automobile industry. The Grand Challenge held by DARPA is a competition of traversing the Nevada desert using autonomous vehicles, and five teams have already succeeded in this event in 2005. Additionally, in 2007, another competition, the Urban Challenge, required completion of the track that consisted of 96 km of artificially composed urban terrain, abiding by all traffic rules within a period of 6 hours. Seven teams, including the teams from the Carnegie Mellon University and the joint team of GE were able to complete it.
The detection of the lanes is the first aspect of the operation of the autonomous vehicle on which computer vision technology can be applied to. Research work on the detection of lanes is divided in two categories based on the type of analyzed data, namely, images [112] or videos [13, 14]. Additionally, there are three divisions that serve the purpose of the research, namely, a lane-departure warning system, lane-attention monitoring system, and the automated vehicle-control system [1]. Because of the applied algorithms, detection of the lanes was separated into the straight-lane sections, and the curved-lane sections. In most cases, research on the straight region, is usually based on Hough transformation, and it combines appropriate methods to increase the detection rate.
Park et al. [2] used a fuzzy binarization method to remove obstacles from the road, shadows, other vehicles, and surrounding environments, and they effectively detected the straight lanes by applying the Canny operation and the Hough transform.
Bae and Lee [3] first estimated a horizontal vanish point and selected an optimized processing area, which is named as the region of interest for lane boundary (ROI-LB). They used the Hough transform by applying nonparametric model matching techniques and thus improved the time of lane detection.
In the research work of Ahn and Han [4], clustering was performed after the application of the Sobel and thinning operations in order to recognize the lane by comparing regions with similar widths within the image, with that of the actual lane. Cho et al. [5] suggested a method where several ROIs were set at low-light situations, such as during the night, or inside a tunnel, and applied a gamma adjustment to the image before the use of a lane-detection algorithm using a multichannel, selective Hough transform. This method exhibited an excellent performance on both straight and slightly curved lanes.
Jang et al. [6] proposed a method to detect the curved lanes with the application of a variable threshold calculated based on the probability distribution of the lane area instead of a fixed threshold value for extracting the edge of a lane. The purposes of these researcher studies focused on the lane-departure warning system, lane-attention monitoring system, and the automated vehicle-control system, as referred earlier.
Unlike these studies, this paper suggests a usage of a lane detection algorithm to eliminate illegal parking of vehicles. In developed countries, the traffic order is well maintained, but this is not the case in Korea. The law and order index, including traffic regulations, ranks 27 among the 34 countries of the OECD. Based on the report which JeJu island has presented in 2013, the economic losses that occurred by illegally parking vehicles amounts to 10.7 trillion Korean won per year [15]. This national expense is ten times higher than the annual social cost of the three most violent crimes, namely, murder, robbery, and rape, amounting to 390.3 billion Korean won [15].
In Korea, illegally parked vehicles are easily found not only in Jeju island, but in most other places in the country. These vehicles increase the travel time for others by interrupting the traffic flows, and by providing a cause for accidental contacts while vehicles change lanes. Currently, to eliminate these vehicle violations, parking enforcement agents conduct surveillances, and once they identify these vehicles, they exit their surveillance vehicle to take pictures for assigning a penalty. This method takes significant time and expenses in the effort to minimize illegally parked vehicles. The number of these vehicles has increased, and the law and order indices are becoming lower than those reported during the past five years [13].
In this study, the regulation of illegal parking is automated using computer vision algorithms for the reduction of seizing time and expenses. Section 2 of this paper explains the overall system and its use for detecting lanes, while Section 3 shows the types of lanes and their distinguishing methodologies. In Section 4, we present the experimental results. Conclusions are outlined in the last section.

### 2. Lane Recognition System

In Korea, when an illegally parked vehicle is discovered, enforcement agents either exit their surveillance vehicles to take pictures of the illegally parked vehicle, or they collect a series of consecutive images using the camera installed on the roof of the surveillance vehicle, as shown in Figure 1. However, the images taken from the roof camera also have to be checked manually. This paper suggests an automated system that can be used to detect illegally parked vehicles using captured images from the rooftop camera.
When a vehicle is captured by a camera, there are many objects that can be recognized in the captured scene, such as vehicles, lanes, and the surrounding environment. There is always shadow underneath the vehicles, although sometimes it is less prominent depending on the intensity of the light. However, it can provide important and useful information. Using this shadow, we recognize the presence of vehicles, and the relative position of the vehicles and lanes can be distinguished to determine parking violations. In this paper, parking violation refers to the parking of the vehicle on the prohibited area, and it can be determined by the type of the lane closest to the vehicle.
Several assumptions have to be defined in regard to the implementation of the methodology of image capture in this study. First, images are only taken during daytime hours. Even though parking enforcement can take place during the night, we need to conduct additional image processing for further use of the images. Therefore, this research targets only the images taken during daytime hours.
Second, the study targets only the lane which is relatively clearly distinguished in the images for the assessment of the parking violation. Lanes can be damaged by the usage of the roads from traffic, or they can be indistinguishable due to foreign matter, especially foreign matter on the sidewalk that is of foremost importance in this research study. Clearly, the cases where foreign matter or debris exists can be analyzed after the completion of image processing. However, since the additional processing involved does not constitute a major methodological concern, only clearly distinguished lanes are targeted.
Third, in many cases, parking violations include several vehicles in a row. Since the vehicle’s row is defined by the type of lanes underneath them, it can be changed from a double yellow solid line to a yellow dashed line. In order to capture the image of the lane, the position of the camera in Figure 1 should be changed to target the lower part of the vehicles. Furthermore, the violation cases that include two vehicles that are too close to distinguish their license plates are excluded from this study.
The entire algorithm is shown in Figure 2. Vehicles are detected from the image taken from the camera, and the region of interest is set to distinguish the lane. The relative position of the vehicle and the lane is considered in order to determine on whether there is a violation.
To identify the vehicle, the shadow is first detected by processing the image using fuzzy binarization, based on its brightness. Among the several shadows that exist in the image, the shadow with the largest size is selected as the shadow of the vehicle of interest.
In order to determine on whether the object is a vehicle, region of interests are set upon the shadow to distinguish the presence of the license plate. After the object is determined to be a vehicle, region of interests are reset to the lower part of the image in order to distinguish the lane type. Color, connectivity, and number of the lanes, are mainly analyzed from the image. With the exception of white solid lines, there exists a possibility of parking violation. The Canny operation and Hough transform are used to calculate the slope of the lane and the relative position of the vehicle. In turn, the road lane is used to determine the existence of a violation.

### 3. Classification according to the Type of Lanes

According to the traffic law, there are three cases which are considered as parking violations, and for which the vehicle can be towed, or a penalty can be charged. Specifically, these cases include the a) case of parking on the lane, which prohibits both stopping and parking, b) the case of parking on the lane, which prohibits only parking, and c) the case of parking overtime on the lane, which is allowed only for a certain amount of time. In this section, the types of lanes that are used in Korea are introduced and the algorithms used for their distinction are explained.

### 3.1 Lane Types

Currently, four types of lanes are used in Korea, and the rules that pertain for each of these are described below:
Figure 3(a) shows the white solid line. As parking and stopping are both possible, vehicle parking on this line cannot be considered as a violation. Figure 3(b) presents the yellow solid line. Parking and short-term stopping are flexibly allowed for a certain amount of time, which is displayed on a secondary sign. According to the Road Traffic Act 34, the parked vehicle can be a target of enforcement based on time.
Figure 3(c) is the double yellow solid line which prohibits both parking and stopping. Thus, parking of a vehicle on this line is considered as an instant violation. Figure 3(d) presents the yellow dashed line. Parking is prohibited in principle, but stopping is allowed for a period of 5 minutes. However, the vehicle parked on this line can be also subject of enforcement according to the traffic law.

### 3.2 Fuzzy Binarization

The general method of binarization is presented in accordance to equation (1). It substitutes a threshold for the average of the brightest pixel and the darkest pixel in the original image. Setting a threshold standard, we can divide them into two groups, namely the group with zero values for the higher intensity pixels, and the group of unity values for the smaller intensity pixels [2].
##### (1)
$T=Imax+Imin2$
In (1), T refers to a threshold, Imax to the pixel of maximum brightness, and Imin to the pixel of minimum brightness. However, it is difficult to generate good results by using the threshold obtained from this method. That is why there is fuzziness in selecting the threshold for detecting the existence of the feature points.
To remedy this fuzziness, the use of a fuzzy binarization method is proposed. However, the existing method uses the entire image dataset, so it is not suitable for an edge-detected image whose feature points were already filtered to some extent. In short, if the pixel value was zero in the edge-detected image, it meant that there was no information at all. Therefore, in this paper, the remaining data, excluding the pixel value 0, was applied to process the fuzzy binarization.
Each of the variables Xr, Xg, and Xb, are defined as a result of the image’s input RGB values, and the calculation of the medium brightness is in accordance to the following equation:
##### (2)
$Xm=Xm+Xir+Xiq+Xib3,ifXir+Xig+Xib3>0 =Xmcount,otherwise.$
We then calculated the distance value of the dark range, Dmin, and the distance value of the bright range, Dmax, by using Xm.
##### (3)
$Dmax= ∣Xh-Xm∣Dmin= ∣Xm-Xl∣$
At this point, Xl is the value of the darkest pixel, and Xh is the value of the brightest pixel in the input image. We also calculated the maximum brightness value, Imax, and the minimum brightness value, Imin, by applying Dmin and Dmax in accordance to the following rules:
##### (4)
$Xm=255-Xmif Xm>128 =XmotherwiseImin=Xm-Xmif Dmin>Xm =Xm-DminotherwiseImax=Xm+Xmif Dmax>Xm =Xm+Dmaxotherwise.$
The brightest value Imax and the least bright value Imin were applied in order to extract a triangle-type membership function.
Therefore, the triangle-type membership function has an interval. In the membership function, we calculated the medium brightness value Imid to make the membership equal to 1, as follows:
##### (5)
$Imid=Imax+Imin2.$
Moreover, the membership of section |Imin, Imax| can be calculated as in (6). We applied the α-cut membership [μ(x)], which was obtained previously in the membership function, in order to implement the image binarization. At this point, we set the value to be equal to 0.5. According to this function, in order to generate a binary image, we set the value of the image pixel to 255 when the membership is higher than 0.5, and when it is lower, the value of image pixel is set to 0. Figure 4 shows the image of the applied fuzzy binarization.
##### (6)
$μ(x)=0, if Xm≤Iminor Xm≥Imax, =(Imax-Xm)(Imax-Imid),if Xm>Imid, =(Xm-Imin)(Imid-Imin),if Xm

### 3.3 Assessing the Presence of a Vehicle

One of the basic assumptions in this study is that the images are captured automatically using the camera mounted on the enforcement vehicle. Therefore, the determination on whether a vehicle is present or not, and whether there is a parking violation, requires sequential processing.
It is not easy to recognize every vehicle in the image. Instead of recognizing the vehicles, the study presents the methodology for the assessment of the presence of a vehicle. First, we determine whether the object exists in the image and whether the object is a vehicle or not. The shadow is used as the standard criterion for this assessment.
The region that defines the shadow is detected using fuzzy binarization, as described in Section 3.1. Of course, there can be several shadows in an image. The object associated with the shadow should be the targeted vehicle that needs to be recognized; however, the recognition process is complex.
There is a variety of car types, such that, multiple models for each type exist that have different shapes, even in the case of a sedan. Therefore, distinguishing the vehicle through a priori learning of the appearance of all vehicle types is cumbersome and ineffective. Instead, the general shape outline of the car could be used to detect the vehicle. However, even the establishment of the general concept itself is an unsolved problem in the field of computer vision.
The assumption imposed by this study is that the enforcement vehicle would move along the lane and capture an image of the target from a close distance. Consequently, the shadow of the target would not be small. Thus, shadows with a relatively small size are eliminated from being considered as the vehicle’s shadow.
The license plate is a common feature of all vehicles, and it is used to determine the target’s shadow. A region of interest is set on an upper part of the shadow in order to apply the algorithm suggested by Rhee and Kim [16] for detecting the license plate and verifying it. Figure 5 shows an image of the license plate extraction, based on the original image, using the algorithm.
After detecting the vehicle, another region of interest is set to a lower part of the vehicle. If the lane is detected in this region, the type of lane is determined using the color, connectivity, and number of the lane.

### 3.4 Classification of the Lanes

As explained in Section 3.1, there are four types of lanes, and based on color, they can be divided into two, namely, white and yellow. Therefore, when the lane is detected, the color of the lane is distinguished first. If the color is yellow, the form of the lane is then recognized to be either a solid, double solid, or a dashed line.
From the region of interest that was placed below the vehicle, the boundary of the lane is extracted using the Canny operator, and the Hough transform is applied to predict the location of the lane by obtaining a straight line from the previous process. At this point, the RGB color form is used to recognize on whether the line is white or yellow. The ability to distinguish the color is enhanced when the lane was first painted, but it diminishes with time due to traffic and road use. To verify the color, an appropriate tolerance is specified in this study, which was estimated experimentally. After identifying the color of the lane, and in the case that the color is white, it is classified as a white solid line. If not, an additional check is executed to determine whether the color is yellow. The HSV color format which is least affected by the brightness is then used to verify the yellow color.
If the color of the lane is confirmed as yellow, we can verify the form of the lane. After the application of the fuzzy binarization to the region of interest, the generated binary image contains the extracted foreground objects. In the case where the extracted object is only one and its location is consistent with the lane extracted previously, we consider it as a yellow solid line. The algorithm recognizes the object as a yellow dashed line if three conditions are satisfied: a) the number of objects is two, b) both exist on the line obtained by the Hough transform, and c) squares that enclose each of the two objects have no overlapping regions. In this case, the algorithm considers that one straight dashed line is drawn, and the determination is thus a yellow dashed line. Lastly, when the number of extracted objects is two, and the squares that enclose each of the two objects are overlapped by a distance (which is determined experimentally) larger than d, the lane type is considered to be a double yellow solid line.
After the type of the lane is confirmed through these processes, the vehicle parked on the white solid line is considered as legally parked, and in case of a yellow line, the determination of a parking violation is decided based on whether the square that surrounds the lanes overlaps with the square that surrounds the shadow of the vehicle.
Since the presence of a yellow solid line indicates that parking is allowed only for a period of time, which is displayed on the secondary sign, parking violation is determined based on the time at which the image is captured. Recognition of the secondary sign is not described in this research work. The case where the vehicle is parked on a double yellow solid line is regarded as a parking violation, since both parked and stopped vehicles are restricted regardless of time. On the yellow dashed line, parking is forbidden, but a short-term stop is allowed for a period of 5 minutes. For official usage of illegal parking enforcement, images of the same location should be captured twice with a 5-minute gap. However, this research study mainly focuses on the recognition of the lane and the determination of its type, so the image is taken only once to allow recognition of the yellow dashed line.

### 4. Experimental Results

The specifications of the hardware used in these experiments are as follows: the CPU was i7-4702mq, the RAM was 8 GB, and Nvidia GT745M was used for the graphics board, while the OS was Microsoft Windows 7 Enterprise 64 bit. The program was built on C++ under the Microsoft Visual Studio 2013 Professional using the OpenCV 3.0 library.
The images which are used for the experiments are taken from a smart phone instead of a camera mounted on the enforcement vehicle moving along the lane, due to the practical difficulties. However, the image capture methodology emulated closely the enforcement vehicle image capture while in motion. The resolution of the acquired images was 1280 × 720. Figure 6 shows the region of interest applied for the detection of lanes.
The number of sample images used was not the same but varied according to the lane type. Table 1 lists the experimental results.
Figure 7 shows the results of detection the white solid line, the yellow solid line, the doubled yellow solid line, and the yellow dashed line. When 50 images were included, the detection ratio of the white solid line is 100%. The yellow solid line is detected in 45 images out of 47 that included it. The detection ratio is 95.8%. The double yellow solid line is detected in 69 images compared to a total of 74 that included it. The detection ratio is 93.3%. In addition, the detection ratio for the yellow dotted line is 88.4%. It is detected in 53 images among 60 tested images. The average detection ratio is 93.9% and the average execution time was 0.367 seconds.
The major reason for the inability to detect the lanes is the increased intensity of the sunlight. It confused us in our effort to classify the vehicle’s shadow, or led to roads appearing too bright to allow detection of the color. In regard to the detection of yellow dotted lines, because the interval between the dots is not regular, we have encountered cases where the two dots could not be found in the image.

### 5. Conclusions

The presented research work introduces a classification method for the determination of parking violations using images captured by a camera. The shadow region was extracted from the image, utilizing information regarding the brightness, and the shadow of the vehicle under consideration was determined using the properties of the region. The ROI was set upon the shadow in order to verify the license plate of the vehicle, and another region of interest was set to the bottom of the vehicle to detect the lane rapidly.
At the ROI, preprocessing was conducted for lane detection, such as for example, the application of morphological operations to restore the damaged lane, and the application of the fuzzy binarization method, Canny operator, and Hough transform.
Whether the vehicle was illegally parked or not was determined by the type of the lane, and the comparison of the relative position of the vehicle’s shadow and the lane. We obtained an average detection ratio of 93.9%, and the elicited average execution time was 0.367 seconds in the conducted experiments. Based on the analysis of the results, we felt the necessity to also develop an algorithm to detect lanes from video datasets.

### Acknowledgements

This work was supported by the Ministry of Education (MOE) and the National Research Foundation of Korea (NRF) through the Human Resource Training Project for Regional Innovation (No. 2015-HICLA1035526).

### Biography

A-Ram Kim received his B.S. and M.S. degrees in Computer Engineering and Advanced Engineering from Kyungnam University, Changwon, Korea, in 2011 and 2013, respectively, and is currently pursuing the Ph.D. degree in Advanced Engineering at Kyungnam University, Changwon, Korea. His present interests include intelligent robot, image processing and pattern recognition.

### Biography

Sang-Yong Rhee received his B.S. and M.S. degrees in Industrial Engineering from Korea University, Seoul, Korea, in 1982 and 1984, respectively, and his Ph.D. degree in Industrial Engineering at Pohang University of Science and Technology, Pohang, Korea. He is currently a professor at the Computer Engineering, Kyungnam University, Changwon, Korea. His research interests include computer vision, augmented reality, neuro-fuzzy and human-robot interface.

### Biography

Hyeon-Woong Jang received his B.S. degrees in Computer Engineering from Kyungnam University, Changwon, Korea, in 2014, and is currently pursuing the M.S. degree in Advanced Engineering at Kyungnam University, Changwon, Korea. His present interests include computer vision, image processing.

### Notes

Conflict of Interest

### References

1. Illingworth J, Kittler J. A survey of the Hough transform. Computer Vision, Graphics, and Image Processing. 44:87–116. 1988; http://dx.doi.org/10.1016/S0734-189X(88)80033-1. DOI: 10.1016/S0734-189X(88)80033-1.
2. Park JW, Shin SM, Kim KB. Lane detection using fuzzy binarization and Hough transform. Journal of the Korea Institute of Electronic Communication Sciences. 5(2):473–477. 2011.
3. Bae TW, Lee SI. An iterative bilateral weighted median filter for the removal of high-density impulse noise. Journal of Korean Institute of Information Technology. 8(2):59–65. 2010.
4. Ahn SJ, Han MH. Research of the lane recognition for an advanced vehicle system. Journal of Korean Institute of Information Technology. 5(1):136–142. 2007.
5. Cho JH, Jang YM, Cho SB. A high-performance lane recognition algorithm using word descriptors and a selective Hough transform algorithm with four-channel ROI. Journal of the Institute of Electronics and Information Engineers. 52(2):148–161. 2015; http://dx.doi.org/10.5573/ieie.2015.52.2.148.
6. Jang C, Lee S, Choi C, Kim YK. Realtime robust curved lane detection algorithm using Gaussian mixture model. Journal of Institute of Control, Robotics and Systems. 22(1):1–7. 2016.
7. Cheong CK. A high speed road lane detection based on optimal extraction of ROI-LB. Journal of Broadcast Engineering. 14(2):253–264. 2009; http://dx.doi.org/10.5909/JBE.2009.14.2.253. DOI: 10.5909/JBE.2009.14.2.253.
8. Woo SB, Park CS, Kim KB. Lane detection using fuzzy C-Means clustering. In : Proceedings of the KIIS Fall Conference; Seoul, Korea. 2009; p. 396–401.
9. Lin Q, Han YJ, Hahn HS. Lane detection in complex environment using grid-based morphology and directional edge-link pairs. Journal of Korean Institute of Intelligent Systems. 20(6):786–792. 2010; http://dx.doi.org/10.5391/JKIIS.2010.20.6.786. DOI: 10.5391/JKIIS.2010.20.6.786.
10. Park T, Cho TH. A crosswalk and stop line recognition system for autonomous vehicles. Journal of Korean Institute of Intelligent Systems. 22(2):154–160. 2012; http://dx.doi.org/10.5391/JKIIS.2012.22.2.154. DOI: 10.5391/JKIIS.2012.22.2.154.
11. Jang GH, Kwak SW, Jung MW. Lane detection algorithm based on color information. In : Proceedings of the KIIS Spring Conference; Mokpo, Korea. 2012; p. 299–300.
12. Lee HS. An adaptive image algorithm for object detection in night drive. Proceedings of the KIIS Fall Conference 2010. 20(2):416–419. 2010.
13. Cheng HY, Jeng BS, Tseng PT, Fan KC. Lane detection with moving vehicles in the traffic scenes. IEEE Transactions on Intelligent Transportation Systems. 7(4):571–582. 2006; http://dx.doi.org/10.1109/TITS.2006.883940. DOI: 10.1109/TITS.2006.883940.
14. McCall JC, Trivedi MM. Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Transactions on Intelligent Transportation Systems. 7(1):20–37. 2006; http://dx.doi.org/10.1109/TITS.2006.869595. DOI: 10.1109/TITS.2006.869595.
16. Rhee SY, Kim YB. Rear car license plate detection of one more cars. Journal of Institute of Control, Robotics and Systems. 12(4):400–404. 2006; http://dx.doi.org/10.5302/J.ICROS.2006.12.4.400.
##### Figure 1
Parking enforcement using a vehicle.
System overview.
##### Figure 3
The types of lanes used in Korea: (a) white solid line, (b) yellow solid line, (c) double yellow solid line, and (d) yellow dashed line.
##### Figure 4
Resulting image after the application of the fuzzy binarization.
##### Figure 5
Detection of the car license plate.
##### Figure 6
Original image and region of interest.
##### Figure 7
The result of the detection of the lanes: (a) white solid line, (b) yellow solid line, (c) doubled yellow solid line, and (d) yellow dashed line.
##### Table 1
The detection rate of lanes
Success Failure Rate (%)
White solid line 50 0 100
Yellow solid line 45 2 95.8
Double yellow solid line 69 5 93.3
Yellow dotted line 53 7 88.4
Total 217 14 93.9
Formats:
Article |
PDF(1.5M) | PubReader | ePub |