If you upgraded to macOS Catalina it does a number of conversions during the install and then during the first time some apps are used. In your case that would be the old (iPhoto) .photolibrary would be converted to the different (but similar) Photos Library.photolibrary format.
And yes you can't just open a .photolibrary file in the Finder and yes it is a directory that macOS treats as a file. Apps are the same kind of thing, a directory (or folder) that contains all of the code, images, configurations and other things that an app needs to run.
A .photolibrary can be opened in the Finder. To do so either Ctrl - click on the .photoslibrary file (in your Pictures folder) or, if you have your trackpad set up to right click or have a two button mouse, right-click on the file (called a "package file") and select "Show Package Contents" from the menu that pops-up.
Inside is a bunch more folders that the photos.app uses to keep track of all the pictures it catalogs. But if you look in the "originals" folder your pictures will be there, but with names that only photos.app understands. So you may have to go through any and all photos there to find the one(s) you need.
And if you look for a "Recently Deleted" item in the Photos that will contain any photos that have been deleted in the last 30 days. So if a picture was deleted more than 30 days ago, it's gone.
And if you want to recover all the space that the photoslibrary takes up you can just delete it. When Photos.app launches again it will create a new, empty, library.
Does that answer your question?
Best Answer
You cannot downgrade a photo library, since downgrading overall is unsupported. Your files are not lost, however, since the Photos library is a bundle that still contains your originals. You have the following options:
If you use iCloud Photo Library, create a new Library and configure it with iCloud. It will pull down your photos.
If you have access to a machine with Big Sur you have two routes:
If you cannot do either of these and must be on Catalina, you must open up the photo library and extract the originals manually. If you locate the library in Finder (Usually located in the Pictures folder) and right click it, click Show Package Contents. There is an originals folder that contains all your unmodified images. You should be able to import this folder.
These options are listed in order of best to worst. Only use approach 3 if none of the other approaches are viable. Even if you don't have a second laptop, considering installing Big Sur on a second volume (here are instructions for the beta, but they apply for the regular release as well) and using that to upload to iCloud Photo Library or Export.