Abstract: In order to integrate the return and delivery service in the process of logistics and distribution, a vehicle routing model with time window is established according to the actual situation, and a Q-learning self-heuristic ant colony algorithm based on K-means clustering processing is designed to solve such problems. According to the characteristics of distribution service, by improving the basic k-means algorithm, the local search ability of the ant colony algorithm is improved, and the reasonable connection of the two algorithms completed. Experiments are carried out according to the relevant literature data and standard examples to verify that the proposed algorithm has good performance and can solve the described problems.