Regular expressions (or simpler splits on [ and ] characters) should work over some inputs, but will struggle a bit with a more general set of possibilities.
Consider, for example:
There is an [innermost [bear] in the courtyards] of the palace.
Nested ( recursive ) patterns are classically beyond the scope of Kleene algebras (regular expressions), though some implementations make efforts to get around that architectural constraint.