I am observing that the right sidebar does not immediately appear. Yesterday the pages where I added headings (markdown #, ## etc) has the right sidebar visible now with proper table of content. But just now I tried adding new headings inside other pages, but there is no right sidebar for them (yet :) ). Looks like the right sidebar is getting created in the background - maybe the cron task needs to be run. I am not sure.. BTW, I cleared cache to check and that has no impact.

The left sidebar I am assuming is manually created with the icon etc manually added. I could not find any block that does this. Maybe someone from the team will reply tomorrow.