He feels Sam stiffen and has a few guesses as to why that happens. But. He'd rather be honest than not right now.

"You love your job; I would never ask you to set that aside for my own paranoia or whatever." He frowns, looks down, shrugs. "I know it happens all the time. I know that there's fuckall I can do about that. And I don't think the shit you get up to, living your life, is inherently dangerous. Broadly, yanno?"

He just thinks that Tony is not the one. And that he, himself, and Sam? Would be great together.