Currently, social media is full of rumors. To stop rumors from spreading further, rumor detection has received increasing attention. Recent rumor detection methods treat all propagation paths and all nodes on the paths as equally important, resulting in models that fail to extract the key features. In addition, most methods ignore user features, leading to limitations in the performance improvement of rumor detection. To address these problems, we propose a Dual-Attention Network model on propagation Tree structures named DAN-Tree, where a node-and-path dual-attention mechanism is designed to organically fuse deep structure and semantic information on the propagation structures of rumors, and path oversampling and structural embedding are employed to enhance the learning of deep structures. Finally, we deeply integrate user profiles into the propagation trees in DAN-Tree, thus proposing the DAN-Tree++ model to further improve performance. Empirical studies on four rumor datasets have shown that DAN-Tree outperforms the state-of-the-art rumor detection models learning on propagation structures, and the results on two datasets with user information validate the superior performance of DAN-Tree++ over other models using both user profiles and propagation structures. What's more, DAN-Tree, especially DAN-Tree++, has achieved the best performance on early detection tasks.