By default, you may set icon only in your own OneSignal account. But also you may change another parameters in the code (using javascript language) if you like to change design. For example you may set individual icons for each notification and it will not work in Safari, but ok for Chrome and FF.

You may read what exactly maybe changed for web push notifications in more details here https://documentation.onesignal.com/docs/web-push-sdk