Produced by KelbyOne

Nesting Smart Objects for Multimask Effects in Photoshop

One of the biggest drawbacks to smart filters is that all filters applied to a single layer must share a common filter mask. Fortunately, there’s a workaround. To make a long story short, you nest one smart object inside another. To keep the long story long, read the following steps, in which I’ll employ a total of four smart filters—two masked, two not—to both smooth and sharpen the detail in a photograph.

[If you’d like to download the image used in this tutorial to practice these techniques, click here. We’ve also included a before and after for you to compare the results. All files are for practice purposes only.]

Let’s start things off with a photograph that requires sharpening. I captured this image with a Leica D-Lux 3 out the window of my car. In the course of touring the back roads of South Dakota’s Black Forest, we were suddenly surrounded by a few hundred mostly peaceful bison, and this guy in particular was close enough to touch (were I endowed with really stretchy Mr. Fantastic arms). The resulting image is soft with striated noise patterns and wandering colors. If I just set about sharpening him, I’d bring out the bad detail along with the good. Fortunately, smart objects provide a more selective if somewhat laborious approach.


Photoshop provides a handful of commands that convert an image to a smart object. But for my money, the most convenient method is to click the little doodad in the top-right corner of the Layers panel and choose Convert to Smart Object. (It works for multilayer images, too.) Go ahead and rename the new layer something meaningful like “Bison.” Now your converted and named image is ready to receive nondestructive smart filters.

To sharpen the creature, choose Filter>Sharpen>Smart Sharpen. Then set the Amount to 350%, Radius to 4.0 pixels, and Remove to Lens Blur. After clicking OK, double-click the tiny slider icon to the right of the words “Smart Sharpen” in the Layers panel and change the Mode setting to Luminosity. This focuses the attention on the detail in the image rather than exaggerating the discrepancies between color channels—always a good idea when sharpening. Leave the Opacity set to 100% and click OK. Tip: Delete the empty filter mask because it just takes up room. When you do need a mask, it’s easier to create it on the fly, as you’ll soon see.

Sharpening did my bison some good, but it also brought out anomalous color patches in the fur. What’s needed is a generous pass of the Median filter. Choose Filter>Noise>Median, raise the Radius to a hefty 50 pixels, and click OK. As a rule, smoothing should be applied before sharpening so drag the Median filter below Smart Sharpen in the Layers panel. Double-click the slider icon to the right of “Median” in the Layers panel. (Photoshop warns you that you won’t be able to preview the effects.) Change the Mode to Color to smooth over the considerable coloring problems, and click OK to render the smoothing and sharpening effects together.

Smoothing out the color works nicely inside the bizarrely (and inaccurately) multicolored bison, but it eliminates all hint of color in the background. What’s needed is a filter mask that exposes the bison and protects the grass. Turn off the Eye icon in front of the words “Smart Filters” in the Layers panel to restore the original image, which we’ll need to build a mask. Click the Channels tab and click the various channels to find the one with the highest degree of contrast between foreground and background. For most people and mammal shots, this is going to be Red.

Duplicate the Red channel by dragging it onto the Create New Channel icon at the bottom of the panel. Double-click the name (in our case, Red copy) and call it “Mask.” When the mask is complete, the bison should be white and the background black. So press Command-I (PC: Ctrl-I) to invert the colors. Next we need to exaggerate the contrast. Choose Image>Adjustments>Levels. Raise the black point value to 70 and lower the white point value to 190 to draw out the bison from its background. Click OK to complete the effect.

Your mask isn’t going to win any awards for accuracy. And if you have some extra time on your hands, you could finesse it using the Dodge and Burn tools or the like. But this quick-and-dirty mask nicely highlights the areas of major color drift and fur detail, making it good enough for our purposes. To load the mask, press the Command key (PC: Ctrl) and click the Mask thumbnail in the Channels panel. Now you’re ready to apply the mask to the filters.

Click the RGB thumbnail in the Channels panel to return to the color composite image. Then click the Layers tab to return to the Layers panel. Turn on the Eye icon in front of the words “Smart Filters.” Then Control-click (PC: Right-click) “Smart Filters” and choose Add Filter Mask. Photoshop limits the Median and Smart Sharpen effects to the interior of the bison and protects the prairie grass in the background.

One of the great things about smart objects is that they make efficient use of file size. This particular image was 15.4 MB flat and 15.4 MB after the addition of the two nondestructive smart filters. It’s only when you start adding masks that the file size balloons. We have two masks, one alpha channel and the other a filter mask, so the file size has grown to 25.7 MB. The two masks are identical, so there’s no reason to keep them both. Go to the Channels panel and drag the Mask channel (not the italicized Filter Mask item) to the Trash. The file size drops to 20.5 MB.

I’m glad to have brought back the colors in the grass. But I’m not so keen on limiting the sharpening effect to the animal. Plus we need to address the horizontal noise patterns at work in the shadow detail. This means applying a couple more filters, neither of which benefit from the existing filter mask. The solution is to create a nested smart object—a smart object inside a smart object—to accommodate the unmasked filters. Back inside the Layers panel, double-click on the Bison smart object thumbnail to open it in a new window (presumably called Layer 01.psb after the temporary file Photoshop creates to accommodate the smart object).

You’ll see the original, unedited version of the photograph on a standard Background layer, with all pixels available for editing. Bring up the Layers panel menu and once again choose Convert to Smart Object. Then rename the layer “You’re in the SO!” This way you’ll know where you are if you accidentally enter the smart object. While this adds to the complexity of the file, quite amazingly, it adds nothing to the file size. In fact, the Adobe engineers I talked to endorse this technique and told me there is currently no limit to how deeply you can nest smart objects.

To downplay the striated noise pattern, choose Filter>Noise>Reduce Noise. Then change the Strength value to 8, Preserve Details to 20%, Reduce Color Noise to 15%, and Sharpen Details to 0%, all of which conspire to smooth away most of the noise without altogether ruining the good detail. Make sure the Remove JPEG Artifact checkbox is turned off and then click OK to apply the filter.

You don’t need the filter mask so drag it to the Trash in the Layers panel. Next we want to add a little bit of edge contrast, in part to sharpen the prairie grass but also to generally enhance the image detail. Choose Filter>Other>High Pass, set the Radius value to 20 pixels, and click OK. Double-click the tiny slider icon to the right of the words High Pass in the Layers panel. Drop out the grays by changing the Mode setting to Soft Light. Then reduce the Opacity value to 50% and click OK. The result is just enough sharpness to offset the effects of the Reduce Noise filter.

Close and save the nested smart object. Moments later, you’ll see the meticulously sharpened result thanks to a combination of four filters working nondestructively aided by two smart objects, one nested inside the other. It takes some effort to sharpen an image with this degree of control and accuracy, but the results are worth the effort. And you can modify the filter settings without fear of harming the quality of the image. Smart objects aren’t perfect, but with some planning and slightly more patience, you can establish flexible compositions that permit you to apply filters and adjust their order and settings in any way you see fit.


  • Pankaj says:

    Very good ! I like it.

  • Rad says:

    Nice and easy.

  • Miguel Almeida Deca says:

    Thank you for this very useful demonstration.

    Smart Objects are tres cool!


  • Heitor Salazar says:

    Real Good!!!

  • baroquedub says:

    Great tutorial – in-depth and detailed but easy to understand. I learned loads about filters, channels and smart objects. Thanks you.

    A quick question… I’ve followed your instructions to the letter – making sure that I used exactly the same values for the various settings in each of the filters but my end result doesn’t look quite your SO_FINISH jpeg. Yours looks perfect (full of fine and subtle detail) and mine looks wrong and burned out; a bit like a badly overexposed colour photograph [ ] Any ideas why this might be?

  • Goga. says:

    Fantastic! Thanks a lot. Beautifully described except for step 10 which I tried other sources to solve it out. I worked on my own image and worked perfectly. I assume we can have more than one SO nested in SO. Right?

  • sergio says:

    Great tutorial to sharpen a picture of a Bison taken that short, with a Leica D-Lux 3, at that resolution, with the light coming from that angle, with those colors and with the camera settings that made that grain, compression, focus, etc, etc…

    I’ll keep it in mind if I ever replicate those conditions… but I’d rather play the lotto since it has more chances of success.

    For any other occasion, those values will make infinite number of different results, far from your soooo specific photo.

    For those who don’t know what I meant, values based on pixels and thresholds based on tones (and so many other things) only work the same than here if you have that density of pixels, with those tones, etc. Same with every other absolute factor/value, and the only thing I see relative there is width to height ratio!

  • ketha Babu says:

    Thank you for this very useful demonstration.
    Ketha Babu

  • bonobo says:

    you would need to divide the parameter numbers above with the ratio of the original image (that you worked with) with the image available to download (which is smaller in size I guess)…
    values being absolute, filters (e.g. smart filter) don’t give nice results when applied with the values given above..

  • Glazemoo says:

    Thanks for this useful tips.

  • Thanks alot for the information , it was very helpful. thx

  • Tim Wemple says:

    Deke is it possible to nest Camera Raw inside a Camera Raw smart object? I would like to mask the noise reduction in Camera Raw inside a Camera Raw smart object. I use multiple layers of Camera Raw set at different blend modes and different exposure settings to increase detail in my photos and I haven’t been able to find a way to mask off the noise reduction inside those multiple Camera Raw smart objects to another Camera Raw file without the noise reduction. This becomes important especially with clothing, blue jeans to be specific. The noise reduction can really play havoc with the grain in the jeans. Hope that wasn’t to confusing.

    Thanks Tim