pypdf: Calling remove_from_tree on outline item causes "Removed child does not appear to be a tree item" error

When I mentioned to my manager that I came up with a way to automate the task of applying complex page labeling to a PDF document (using pikepdf), he asked me if there was an easy way to change bookmark (aka outline item