This seems to be fast enough, and since I often find myself aborting a current backlink insertion query because I missed to include all Org mode buffers, making it the default only seems to be reasonable.