The open telescope design of the James Webb Space Telescope (JWST) allows light from off-axis sources to scatter into the instrument field of view. The significant sources of stray light in the near IR and the mid-infrared waveband are galactic light and reflected sunlight and thermal emission from the zodiacal dust. The stray light from these sources was calculated with the ASAP software. Backward ray tracing was efficiently used in the prediction of the stray light from the sky. Since the galactic and zodiacal light is distributed over the whole sky, the sky was divided into 7200 patches of size 3 degrees by 3 degrees, and the contribution from each patch was calculated. The instrument geometric susceptibility for each sky patch was calculated with backward ray tracing. Multiplying the geometric susceptibility and the sky radiance, we are able to calculate the stray light from each sky patch. Total stray light from the full sky is then calculated by summing the individual patch contributions. The stray light from the galactic sky and zodiacal light has been calculated for different orientations of the observatory relative to the sky.