If you're seeing errors in the Network panel like "The resource was requested insecurely." or "An error occurred trying to load the resource." originating from XHR/Fetch requests that work most of the time, you may be suffering from a window.stop bug.
When adding an existing Fabric.js object to a group be sure to remove the object from the canvas before adding it to the group. Otherwise, the original object will still be draggable and will muck up the group.