Stimulus money isn’t taken immediately and directly out of the economy: it usually comes from debt. This has a number of downsides (increased debt-service payments and long-term confidence problems are the first two that come to mind), but fiscal stimulus isn’t identical to robbing Peter to pay Paul.
Fiscal stimulus is supposed to work in the short term.
Stimulus spending needs to target real unemployment (which, if you believe the recalculation story, should make you wonder whether stimulus spending actually helps in the long run). Dumping a bunch of money into sectors that’re doing reasonably well isn’t going to magically create jobs in shrinking sectors by vague handwavey second-order effects.