If this is something you envision as its own language, maybe instead of having a tag inside string literal itself you can think of using tag prefix before opening quote in literals, like in Python or Scala, so your examples become
As about ways to improve shell scripting in general, I have found that for the majority tasks I prefer to use a typed programming language instead. For example, recently I needed a throwaway program to process occurrences of regular expression matches in a bunch of files, which is a task that sounds to be well-suited for a throwaway script. I, however, realized that it would be easier for me to write a program in Kotlin instead, which turned out to be a better choice since it was faster to write because I've had access to IDE with code completion and contextual documentation. The resulting program turned out to be easier to debug as well.
This might not work as well when you want to wire up a bunch of external processes, but I think it should be possible to write a small library of helper functions to make that easier as well.