Vocalist Jordan Gatesmith came up with the idea for this Psychedelic-Noise track with guitarist Ian Nygaard. The Howler frontman told Words For Music : "We all switched instruments – I play drums on it and Ian plays sitar on it; but you can't tell its sitar because he puts it through so many noise pedals. So that's just a Noise-Rock track."
The album is titled after this song. Gatesmith explained to HMV.com : "Ian (Nygaard, guitars) lives in this part of Minneapolis where there's this dilapidated church called 'Joy World', which is really really ironic given the state it's in. We all thought it was really funny and a cool expression to name the record."