Does Removing a Task from the Timeline Delete It?

No. The × badge on a task bar only sets Show In Timeline to No. The task remains in the Gantt and task list. See Remove from Timeline.