DeleteCases[set, {___, 0}]
{{0, 1, 1, 0, 1}, {1, 0, 1, 1, 1}}
In this case Pick also works:
Pick[set, Last /@ set, 1]
{{0, 1, 1, 0, 1}, {1, 0, 1, 1, 1}}
You asked about positions other than last. It is possible to generate a working pattern using Blank, e.g. {_, _, 0, _, _}, and this process can be automated, but that is not usually the first approach I would recommend.
You can use individual part extraction in either Select or Cases/DeleteCases, or you can create a "mask" for use with Pick as I showed above. Notable differences are that Select only operates at a single level, while Cases and Pick generalize to deeper structures. Pick is often somewhat faster than other methods, especially if one uses fast numeric functions where possible.
Picking according to the second column:
Pick[set, set[[All, 2]], 1]
{{0, 1, 1, 0, 1}, {1, 1, 1, 1, 0}}
Suppose however that your keep elements are not all the same, like 1 in this example above:
set2 = {{3, 1, 2, 1, 4}, {4, 1, 2, 2, 4}, {2, 3, 0, 0, 1}, {3, 1, 0, 4, 4}};
Pick[set2, Unitize @ set2[[All, 3]], 1]
{{3, 1, 2, 1, 4}, {4, 1, 2, 2, 4}}
Note here that Unitize is needed to make all keep elements into 1. While it is possible to use e.g. Positive instead this is not as efficient because in Mathematica Booleans True and False cannot be packed, while machine-size integers can. It is for this reason that I do not recommend these apparent equivalents:
(* Not recommended where performance matters *)
Pick[set2, Positive @ set2[[All, 3]]]
Pick[set2, set2[[All, 3]], _?Positive]
Pick[set, set[[All, 2 ;; 3]], {1, 1}] but that doesn't recognize the pattern in the last argument – cleanplay Jan 10 '19 at 19:26