Background Low back pain (LBP) is a common health problem experienced after natural disasters. LBP is often concurrent with other musculoskeletal pain; however, the effects of preexisting musculoskeletal pain on LBP are not clear. The purpose of this study was to elucidate the influence of other musculoskeletal pain on new-onset LBP among survivors of the Great East Japan Earthquake (GEJE). Methods A longitudinal study was conducted with survivors at three and four years after the GEJE (n = 1,782). Musculoskeletal pain, such as low back, hand and/or foot, knee, shoulder, and neck pain, were assessed with self-reported questionnaires. New-onset LBP was defined as LBP absent at three years but present at four years after the disaster. Musculoskeletal pain except for LBP at three years after the GEJE were categorized according to the number of pain sites (0, 1, ≥ 2). Multiple regression analyses were performed to calculate the odds ratio (OR) and 95% confidence interval (CI) for new-onset LBP due to the other musculoskeletal pain. Results The incidence of new-onset LBP was 14.1% (251/1,782). Other musculoskeletal pain were significantly associated with new-onset LBP. Using “0” as a reference, the adjusted OR and 95% CI for new-onset LBP were 1.69 (1.17-2.42) in ”1” and 2.85 (1.86-4.36) in “≥ 2” musculoskeletal pain (p < 0.001) Conclusions Preexisting other musculoskeletal pain was associated with new-onset LBP among survivors in the recovery period after the GEJE. Attention should be paid to other musculoskeletal pain sites to treat and prevent LBP after natural disasters.