How best to combine object detection and tracking

I am trying to make a computer vision system which will be able to detect and track objects of interest. This will require (1) detection functionality to notice the object when it appears (2) classification functionality to know the object is indeed of the correct object class and (3) tracking capability to follow the object. Also importantly, the pipeline will need to process huge images (~25megapixels) in close to real time. I am trying to find the best data/algorithmic pipeline to accomplish this. I know for example I can just use a deep network for oject detection, which will also implicitly do classification and tracking. However, at 25megapixels that will run extremely slow. Initial Approach: I am thinking to use some very fast algo for background subtraction to identify putative patches of the image which are changing. If I make the assumption the camera is stable, I can just use vanilla background subtraction. Otherwise, I am not sure how to do this. The putative patches will go to a fast deep NN for classification (such as MobileNetV2). The patches which are identified to be of the class of interest will be sent to a correlation filter based tracker such as SiamFC . In summary: background subtraction --> classification --> tracking. Note only background subtraction operates on the whole image. The other 2 only operate on local patches. Is this a sensible approach? Are there any better approaches? Also, does this problem have a name? I havent been able to find anythi…