angular-material-expansion-panel | Material Design Expansion Panels for Angular Material | User Interface library

I'm working in an Angular 11 project with Material.

I'm using some expanding panels. I'm going to have a button in the header of an expanding panel. Because of this, I want the row to expand/collapse only when the arrow icon (on the right) is clicked. Right now the panel will expand/collapse if any part of the header is clicked.

I found this posting: Angular Material Expansion panel, expand only on button click
However that person's solution is to have their own button with an arrow icon rather than use the arrow icon that material panels already have. I would much rather use the icon that material provides in the panel than create my own button.

Is there some way for me to signal to material that I only want to expand when the toggle icon is selected? A property or something? I would think that many people would want to have type of functionality...
If not, is there a way for me to force this behavior? I don't want to create my own toggle button, I'd want to use the toggle in the panel.

I made this demo, showing my issue: https://stackblitz.com/edit/expand-collapse-on-icon-click-only?file=src/app/expansion-expand-collapse-all-example.html

Any help would be appreciated.