the dubious estimates nH Predict spits out seem to be a feature, not a bug

This is the major problem with algorithms, one of the issue being that they will produce a lot of false positives even if there are best intentions.

But another major problem is that you can influence the outcome by altering the parameters as the article also says. We have been observing similar issues in health and social policy in many countries over the last years, and the results have always been devastating. And research suggests that biases may increases dramatically in the future if we continue to use these algorithms the way we do it now.