Abstract: UAV trajectories are the foundation of UAV distribution route planning. The two are interdependent, where local optimization does not necessarily guarantee global optimization. A bi-level cooperative planning model of UAV distribution routes and trajectories with integrated loading and unloading is established based on GIS rasterization and considering the diversity of customer demand. The model aims to find the optimal coupling relationship between UAV trajectories and fleet scheduling cost considering the spatial distribution of scheduling centers, customers, obstacles and high fall-cost cells. According to the characteristics of the problem, a bi-level Q-Learning algorithm embedded with A* algorithm is designed to solve the problem. At the first level, Q-Learning is used to complete the order distribution and generate UAV distribution routes. At the second level, A* algorithm is embedded to find trajectories between any two points visited by UAVs. The solution of the bi-level model is obtained by the proposed algorithm. Finally, a real case is used to verify the effectiveness of the model and algorithm, by calculating the optimal scheduling plan, analyzing parameter sensitivity, and comparing the solving performance between heuristic algorithms and the proposed algorithm.