Looking at the graph it's almost certainly USB noise. Have you tried another, better PSU? Alternatively, an extra ripple filter capacitor (2200 uF, 10 V) on the 5V rail could help too. In general, these USB mics have very little power filtering which makes them extremely susceptible to noise on som...