I always thought of Kryptonite radiation to being akin to some other radiation---it's often not the initial radiation that kills you, but the secondary particles it forms upon collision.
Kryptonite radiation probably passes right through us, like neutrinoes do, without even pausing. But when they meet an "invulnerable" Kryptonian, much harder than the objects around it, it splits into seconary particles that are deadly to the Kryptonian.
Which would explain why the Silver Age Kryptonians weren't vulnerable to Kryptonite until they were rendered super-powered.
Ahhh, a very, very interesting idea. One obvious hole that has always existed whenever someone tries to account for why Kryptonite is so deadly, is this: why is it that Kryptonite is only deadly to Superman and others when their powers are in operation? Kryptonite is harmless, for instance, to those inside Kandor, the City in the Bottle.
If Kryptonite is only made dangerous on reaction with superpowered matter, it makes sense that it ought to be dangerous to Kryptonians only and harmless to the rest of us: they pass right through normal matter, but not to the superdense kind. For this reason, it is hard to accept the explanation present in SCIENCE OF SUPERMAN for how Kryptonite works: it is a radioactive element whose specific radiation is such that Kryptonian cells are not opaque to it. If this is true, it would affect Kryptonians whether they were powered or not.
One of the unintentionally funniest Kryptonite moments has to have been in one of the Superman/Batman team ups in WORLD'S FINEST, where Superman casts off Batman for a mysterious "Powerman."
BATMAN: "Kryptonite is DEADLY to Superman, but not to NORMAL people like US!"
...he says, as he pushes his way past his giant Penny and Chess Set to his car shaped like a bat.