The rigid structure found outside the plasma membrane that surrounds and supports bacterial cells is called the cell wall.