remove background from my images

Hi! I tried some background remover AI (like remove.bg) before to remove background from my images and changing to plain white but currently they are getting too slow. I need around 100-150 images per day so I need a faster solution. I found this tool that works for me - https://removal.ai/. Any more suggestions!

Comments

  • premiere_studios
    premiere_studios Posts: 119 Enthusiast

    slow, but mask in hitfilm?

  • EvilDaystar
    EvilDaystar Posts: 301 Enthusiast

    That's not something you can do super fast but:

    Photoshop

    I hate recommending Adobe products (I hate subscription services) but it has a really great background removal function.

    Affinity Photo

    GIMP 2.10


  • Dimipapa
    Dimipapa Posts: 386 Enthusiast

    Sound like you are asking for rotosscoping. This is a very long fitful process. The best way I know how is with after affects, and believe me I don't want to shill for adobe, but they have something called the roto brush which has recently been improved to make it even better. You might get the 2 weeks free trial and check it out.


  • EvilDaystar
    EvilDaystar Posts: 301 Enthusiast

    @Dimipapa I believe he's looking at removing the background on stills. If you go to  https://removal.ai/ you can see it's for still only.

  • Dimipapa
    Dimipapa Posts: 386 Enthusiast

    @EvilDaystar Wow that works pretty good, why can't they do that with video?

  • TheBenNorris
    TheBenNorris Posts: 2,033 Staff

    @Dimipapa you would not consistent results if you applied a machine learning model trained on single images to a video. Every frame would be adjusted separately, with no knowledge of previous frames. If you did this, you would most likely find that from frame to frame the results would differ greatly, since in reality we, as humans, get a lot of context on what we can see based on what has just happened. That's a basic overview of why image/video solutions require different approaches anyway.

  • Triem23
    Triem23 Posts: 20,070 Ambassador

    @Dimipapa video becomes much more complex because varied elements would move over time and probably generate inconsistent results. Closest thing to that for video is After Effects Rotobrush 2 paired with Content Aware Fill 2.

    Note Rotobrush 1 required manual cleanup. Content Aware Fill 1 was also inconsistent. Took a few extra years for Adobe's hundreds of coders to really make it work.

  • NormanPCN
    NormanPCN Posts: 4,081 Enthusiast
    edited October 2020

    Every frame would be adjusted separately, with no knowledge of previous frames. If you did this, you would most likely find that from frame to frame the results would differ greatly,

    I'm with Ben on this one. While one frame can look "perfect', when multiple frames are played in succession you will see pixel crawl/creep/shimmer due to the slight, maybe very slight, differences from frame to frame. Differences you will not see in stills you will see when played in rapid succession, which is what video is after all.

    Years ago I had this happen with an image sequence cleanup I tried using Photoshop. Each still was "perfect" but the video playback was crap. At first I was huh?, but then after thinking it was duh. Would have been lucky if it worked. I jumped in without thinking.

  • Dimipapa
    Dimipapa Posts: 386 Enthusiast

    Well I mean if it treats every frame as an individual picture though. Rotobrush was the only reason I held on to after affects as long as I did.

  • Triem23
    Triem23 Posts: 20,070 Ambassador

    @Dimipapa Most likely, for now, it will treat each frame as an individual picture. Remember, in general the computer has no idea what the content of a frame might be - it's all just dots. It's only in the last couple of years that we've started to see the rise of "AI-enhanced" or "Machine-learning" tools. Those work with databases of thousands or millions of images. It's relatively easy to define a face based on two eyes, a nose and a mouth that are going to fall within a fairly narrow range of sizes and proportions. From there a general shape can be extrapolated (two arms, two legs, and a torso), and from THERE, edge detection and further analysis can (usually) find the shape of a subject against a background.

    But, for video, this becomes several orders of magnitude more complex, simply because things are going to move. The computer doesn't know if those moving green pixels in the BG are tree leaves in the breeze, grass waving (similar, but very different), or Kermit the Frog going "YAAAAAAY!" At 24-to-60-or-more images per second the needed computation just isn't there... yet...

    Rotobrush is a great example. What's happening there is the user is inputting lines that are effectively eliminating most of the AI work. The AI is being told to basically limit it's search to within the width of the brush line, using a rule that anything inside the close shape stays and everything outside goes away. Rotobrush 1 still required a lot of manual cleanup, Rotobrush 2 is much better, but it's still relying on the user input to tell the computer "This is the stuff I want to keep."

    Full-AI is years (maybe a decade?) away, but it will get there, eventually.

  • EvilDaystar
    EvilDaystar Posts: 301 Enthusiast

    @Triem23 deep fakes have shown us it is possible but that technology is brand spanking new at the enthusiast level and serves a REALLY specific niche.

    Just a few years ago, doing a full realistic face replacement on consumer grade hardware? You'd be laughed at and yet we have people doing better de-aging videos on their home PC's now than million dollar vfx shots.

    Just take a look at Justice League mustache replacement.

    Or the Irishman

    The results are mind blowing.

    Or Corridor Digital replacing the horrible Scorpion King CGI Rock face with a deep fake.

    Not sure where I was going with this ... hopefully it's interesting and relevant?

  • Triem23
    Triem23 Posts: 20,070 Ambassador

    @EvilDaystar Yup, interesting and relevant

    But Deepfake is ALSO limited in that it's not doing a full head replacement - it's replacing the eyes, nose and mouth within the confines of the original cheek and chin bone structure. In all three examples you've listed, it's replacing the original actor with himself.

    Let's look at the infamous "Tom Holland/Robert Downey Jr in Back to the Future" deepfake. (Created by a Hitfilm user)

    Tom Holland doesn't look like Tom Holland, he looks like Michael J Fox because he still has Michael J Fox's chin. I had no clue from this thumbnail it was supposed to be a Fox/Holland replacement. Yeah, I immediately get RDJ....

    And this deepfake:

    Does NOT Look like old Leonard Nimoy acting alongside young Leonard Nimoy. It's still Zach Quinto's cheeks and chin. This is as bad of a nightmare as the original mustache replacement in Justice League.

    Again, Deepfake is a limited AI looking for very specific things - nose, eyes and mouth - and fitting just those elements into the original face. But it's only TRULY convincing when mapping the original actor over him/herself for purposes of de-aging or facial hair changes. I'm looking at the thumbnail for the Star Trek deepfake, and, I'm sorry, that does NOT look like young Leonard Nimoy. Nimoy's entire face was narrower and his jawline had a different shape.


  • EvilDaystar
    EvilDaystar Posts: 301 Enthusiast
    edited October 2020

    @Triem23 That Back to the Future deep fake is flipping amazing! :)

    I did mention that it was a very niche use but a couple years ago this was pure science fiction or required weeks of careful work from talented CGI artists.

    Maybe background removal could be done with a mixture of current background tech for still and compression algorithms like predictive frame and bi directional predictive frames where the AI would manually isolate the foreground on 2 frames and then use those same (or rather similar) predictive algorithms to ease the movement between them?

    Right now, automatic background removal at the consumer level sucks ... anyone ever tried background replacements in zoom or msteams? Pretty bad right? But technology is going so fast ... who knows.

    EDIT: I think if we got RDJ and Holland to do ADR for this scene, it would sell it off way more.

  • EvilDaystar
    EvilDaystar Posts: 301 Enthusiast

    On a side note, Deep Fakes could have a huge impact on how body doubles are used. Right now when you have a stunt person usually they avoid showing the face and cut to another take with the actor after the stunt but now they could do a proper stunt and do a face reveal in a real one shot with little or no trouble.

  • EvilDaystar
    EvilDaystar Posts: 301 Enthusiast

    Actually ... funny we were talking about this but look what craziness Nvidia is coming up with.


  • Triem23
    Triem23 Posts: 20,070 Ambassador

    @EvilDaystar oh, it's going to get there. It's absolutely amazing what can be done with free software on an under $1000 computer these days. But a Deepfake isn't as good as, say, the head replacements in Kung Pow: Enter the Fist. Of course that film required entire teams to do background replacement, tracking, comping, etc. That was 2002. By about 2013 Mocha Pro and Hitfilm hot the costs down from hundreds of thousands to a thousand dollar laptop and a thousand dollars of software. By 2022 we MIGHT have free AI tools to do it all, and we CERTAINLY will by 2032.

    Stunt work will help, but the first digital face replacements for stunt performers I can think of was Titanic in 1997.

    Oh, and there's an insane shot in one of LotR films - Two Towers? - where Legolas grabs a galloping horse and swings up into the saddle. Orlando Bloom replaced mid-shot with a full digital double in a single take in slow motion (so no motion blur to hide the switch)!

    But, yeah, Deepfake and related tech (EBSynth, anyone?) are rapidly replacing days of work by teams of specialists with drag-and-drop-and-wait.

    Just to age myself, back around 2000 or so I was working on an audio design project and my partner at the time was complaining while we were waiting for the computer to rip and tag elements from CDs inside this brand new program called Sound Forge, and I said to him "Dude, two years ago we'd have to have manually play this into a hardware sampler and manually name everything. We're spoiled!" A year later Vegas came out and we could do edit NON-DESTRUCTIVELY for the first time. By 2004 I'd replaced racks of outboard gear and patch bays with a laptop. Got to the point where I'd be sitting in a theater rehearsal building cues during the run through instead of taking copious notes back to the studio!

    How far we've come. 😊

  • EvilDaystar
    EvilDaystar Posts: 301 Enthusiast

    @Triem23 Just go and look how the original lightsabers and spaceship models were done for Star Wars or how the compositing was done for Back to the Future ... now we can do that on consumer grade PC's with software that is free or cost a couple of hundred dollars (Blender or HitFilm).


  • tddavis
    tddavis Posts: 5,096 Moderator

    @Triem23 While not exactly stunt performer, the most chilling example (and the one that made my internal alarm bells ring off the hook) was in 1993's In the Line of Fire when they replaced one of JFK's bodyguard's face with Clint Eastwood's. Made me sit up and take notice that the old adage of, 'believe none of what you hear and only half what see' was done.

  • Triem23
    Triem23 Posts: 20,070 Ambassador

    @EvilDaystar @tddavis Yup and yup.

    Incidentally, Evil, you need to hang out here more often. You always bring good discussions and insights and your tips and assistance are pretty much always dead on. 😊 Been good interacting with you this week. It had been awhile.

  • Dimipapa
    Dimipapa Posts: 386 Enthusiast

    @Triem23 I dunno if I want full AI to come along ever, but I am loving AI enhanced stuff. Even just the colorization effect on Vegas Pro 18, it works surprisingly well with just a little adjustment.