How can I separate the dress from the background?

Question

img=Import["https://i.stack.imgur.com/NjyIK.jpg"]

The dress is closely similar with the color of the background stair rail.

How can I binarize the image to get the dress mask or human mask?

Any methods of Binarize is acceptable, like RegionBinarize, training an semantic segment model

I've tried several Binarize* functions, and EdgeDetect functions, and confirmed that there are some challenges in the right-down part of the dress.

Background explaination: Why to segment? For example, consider there are many dress model pictures, we can cluster them by dress, and remove background we could get better features, for example textures and shapes.

upadte:

Maybe someone knows more about traditional segment methods and has various experiences. Thanks for @rhermans 's comment.

RemoveBackground[Import["https://i.stack.imgur.com/NjyIK.jpg‌"], {"Foreground", {RGBColor[ 0.9157048167754652, 0.8904987221508752, 0.8709273742640429], 0.062}}]

ImagePartition is a good idea with better region effect in Watershed.

WatershedComponents[,mask]//Colorize

I always don't think there is any efficient-universel method to do this. — yode, Jul 13 '17 at 07:05
Do you want the left forearm and hand in front of the dress removed as well? — m_goldberg, Jul 13 '17 at 07:38
Also, are you concerned only with this particular image or are you looking for general method that would on other images with a similar problem? — m_goldberg, Jul 13 '17 at 07:41
I think one way would be to define a Mask manually (related How to retouch (smart fill) photo image?). Or you can try splitting your image into segments for better contrast img = Import["https://i.stack.imgur.com/NjyIK.jpg"];Grid[Partition[EdgeDetect[#] & /@ (ImagePartition[img, 50]~Flatten~1), 4], Spacings -> {0, 0}] — Sumit, Jul 13 '17 at 09:05
Do you need a fully automated procedure or can you afford doing steps by hand? I think you should [edit] your question and explain better what you need. As it stands, the question is too broad. — rhermans, Jul 13 '17 at 10:25
RemoveBackground[Import["https://i.stack.imgur.com/NjyIK.jpg"], {"Foreground", {RGBColor[ 0.9157048167754652, 0.8904987221508752, 0.8709273742640429], 0.062}}] — rhermans, Jul 13 '17 at 11:06
@yode, I don't think is possible to separate the dress from the rest of the white background without working "by hand", the code I put was as far as I think make sense to go, unless the question is better defined. — rhermans, Jul 13 '17 at 15:07
@m_goldberg yes, without hands and arms, I found it's easy to segment the skin area. — HyperGroups, Jul 13 '17 at 15:27
@m_goldberg The datasets are limit in model dress pictures here. May be a particular method suit to the particular image can be generalized to more similar background pictures. — HyperGroups, Jul 13 '17 at 15:34
@Sumit ImagePartition is a good idea! much better for Watershed in global image. — HyperGroups, Jul 13 '17 at 15:51
@bills see update, I found it's hard, and then it could be an good excercise. — HyperGroups, Jul 13 '17 at 15:54

score 4 · Answer 1 · answered Jul 13 '17 at 18:49

My take at this—not perfect by any means.

The main feature I would exploit is that the area you want to segment is spatially well defined, so you can combine the color information with the position one.

Let's first define a white mask:

white = Binarize[ColorDistance[i, Darker[White, .1]], {0, .1}]

and a white stripe inside the image boundary:

border = ImagePad[ImagePad[0 i, -60], 60, 1];

We can now combine them in a backgroundMask and a foregroundMask

backgroundMask = Erosion[border + ColorNegate[white], 3];
foregroundMask = Erosion[ColorNegate[border], BoxMatrix[{60, 30}]] * white;

{backgroundMask, foregroundMask}

and use them in GrowCutComponents to assign the remaining pixels.

mask = Image[
   GrowCutComponents[i, {backgroundMask, foregroundMask}] - 1
];

The result is not perfect—no spoilers here—but almost all the extra white has been removed

SetAlphaChannel[i, Blur[Erosion[mask, 2], 1]]

You can tweak the code to make the boxes a little more asymmetric and improve the segmentation area, but—as others have said—there's a limit to where automatic (non semantic) segmentation can take you.

MaxDetect[img, .2] will give your white – yode Jul 13 '17 at 19:09 — yode, Jul 13 '17 at 19:09
@yode only for pure white though—GrayLevel[1]. – Batracos Jul 13 '17 at 19:12 — Batracos, Jul 13 '17 at 19:12
see my answer, the Neural Network is very useful – HyperGroups Aug 09 '18 at 10:49 — HyperGroups, Aug 09 '18 at 10:49

HyperGroups · Answer 2 · 2018-08-14T08:12:30.373

4

Cool result of NetModel of 11.3.

You can download the models like Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO Data from Wolfram Netmodel Reposity

Open the Notebook, then import the sample image, and run the code.

img=Import["https://i.stack.imgur.com/NjyIK.jpg"]
result[img]

edited Aug 14 '18 at 08:12

answered Aug 09 '18 at 10:05

HyperGroups

8,619
1
26
63

@C.E. I've added an link, you can download the example notebook there – HyperGroups Aug 09 '18 at 10:12
@C.E. OK, I think in this example, it's needless to copy codes from official help examples into answer. – HyperGroups Aug 09 '18 at 10:47
@C.E. Comment cann't show image. This method solved my question greatly, I think its useful for many other people. – HyperGroups Aug 09 '18 at 11:04
@C.E. The most useful point is one know there are some NN models in Mathematica 11.3 to do such an thing. – HyperGroups Aug 09 '18 at 11:06
@C.E. How about now? is it like a answer? – HyperGroups Aug 14 '18 at 11:00
1

Using an image segmentation network is a pretty good idea; I don't see what the problem is with this answer. However, it would be better to just give the complete code that downloads the network from within a notebook. I.e., resource = "Ademxapp Model A1 Trained on PASCAL VOC2012 and MS-COCO Data"; ResourceObject[resource]; net = NetModel[resource]; net[img]. Furthermore, it seems like this network doesn't get you all the way to the final answer, though it's definitely useful as a first step in defining a mask for further segmentation analysis. – Sjoerd Smit Aug 14 '18 at 13:07
I deleted my comments because there is no reason for them to forever mar this answer. I still believe that this answer could be better though, Sjoerd said it pretty well. – C. E. Aug 14 '18 at 17:13

How can I separate the dress from the background?

upadte:

2 Answers2

Linked