The kinetics of the assembly of semiflexible filaments through end-to-end annealing is key to the structure of the cytoskeleton, but is not understood. We analyze this problem through scaling theory and simulations, and uncover a regime where filaments' ends find each other through bending fluctuations without the need for the whole filament to diffuse. This results in a very substantial speedup of assembly in physiological regimes, and could help with understanding the dynamics of actin and intermediate filaments in biological processes such as wound healing and cell division.