In America, we've had a long history of people in positions of privilege and power who argue that innate biological differences between races or sexes explain why our society is unequal. Such people couch these arguments in the language of unflinching rationalism — they are merely acknowledging facts about human nature, facts they say may be uncomfortable, but which are firmly grounded in scientific evidence. Until our society is willing to accept the evidence, the argument goes, we won't effectively address the problem of inequality. "Once we acknowledge that not all differences are socially constructed or due to discrimination, we open our eyes to a more accurate view of the human condition which is necessary if we actually want to solve problems," wrote James Damore in his now-infamous Google memo, in which he argued that women are biologically less suitable for certain jobs in technology.

Those who make these arguments are trying to claim scientific high ground, but their assertions are the product of speculation and confusion about core biological concepts. Since at least the early 20th century, scientists have tried to knock down these flawed arguments, but clearly many people still believe in the inherent biological superiority of whites or men or both. In our current political moment, these beliefs have come to the foreground, as white supremacists openly talk about their genetic superiority with reporters from national magazines, and march through the streets of American cities carrying the symbols of racist regimes, chanting that "Jews will not replace us."

It is thus important to review why claims that biology contributes to racial and gender inequality are so often wrong. Here are four key facts about biology, sex, and race to keep in mind when you hear someone argue that biology explains why women or racial minorities don't enjoy the same pay, careers, or education as white men:

1. Most genetic differences among humans don't break along racial or sex lines.

Modern humans evolved in Africa, and, from there, colonized the rest of the world. As humans settled the different continents, they encountered new climates, diseases, and foods. These environmental differences created an opportunity to produce genetic differences, as local human populations evolved by natural selection to adapt to their non-African homes. In theory, this could have led to substantial genetic differences among geographically distant human populations — if natural selection was strong, and if these populations remained largely isolated from one another. And indeed, some of our most obvious physical traits, like skin color and facial features, have a lot to do with which continent your ancestors are from.

But it turns out that most of the genetic differences among races are really only skin deep. One of the most important scientific findings in recent decades is that the vast majority of genetic diversity among humans is shared by all races. In other words, there are very few genetic variants — distinct versions of a gene — that are present in one race while completely absent from another. In one of the most definitive studies to date, researchers found that, out of 89 million human genetic variants analyzed, less than 1 percent of these were rare in one geographical population but common in another. A different study put it more vividly: If we somehow discovered a new continent, and that continent was populated by "an African nomadic tribe," that tribe "would preserve 87 percent of the worldwide human genetic variation."

That's not to say that biologically meaningful differences among human populations don't exist. Tibetans, for example, are genetically adapted to living at high altitudes. Scientists continue to search for, and find, examples of human genetic adaptations like this one. But to prove these cases requires a lot of evidence. Most claims that racial differences in ability or temperament are due to genetics, are really nothing more than speculation — the evidence is simply not there.

When it comes to differences between the sexes, genetics has little bearing. The only genetic difference between men and women is the male-specific portion of the Y chromosome. The vast majority of the millions of distinct DNA variants among humans aren't on the Y chromosome, and are thus shared among men and women. This means that genetic studies of socially important traits, such as educational attainment, have little to say about why such traits might differ between men and women.

2. Human populations have always been fluid.

Human history is one of migration and mixing. The groups of people that settled in different parts of the world weren't completely isolated from one another. People often moved around, and whenever there was an opportunity to mix gene pools, the evidence says that people took it. Bantu-speaking people from what is now Nigeria and Cameroon migrated into much of sub-Saharan Africa within the past 4,000 years, and reshaped African genetics. Bronze-age civilizations mixed with, and sometimes replaced, hunter-gatherer tribes in Eurasia and East Asia. Europe experienced at least three major waves of migration: the earliest hunter-gatherers, who interbred with Neanderthals; a wave of farmers from what is now eastern Turkey, who largely replaced the earlier Europeans; and a migration of herders from near the Black sea, who brought Bronze age culture and technology to western Europe. Because of such continuous gene flow among humans, the notion of "pure" racial groups — an obsession of white supremacists — has little biological meaning.

3. Showing that genetics affects a trait doesn't prove that racial differences are genetic.

Confusion over the concept of "heritable" traits is probably responsible for most of the misguided claims made about innate biology and social inequality. The flawed arguments usually proceed like this: Studies have shown that trait X (such as intelligence, educational attainment, a propensity to violence, etc.) is heritable (meaning it is substantially influenced by genetics). Therefore, racial differences in trait X must be due to genetics. To see why this argument is wrong, it's critical to understand what scientists mean when they say that something is heritable. The crux of it is this: When scientists measure heritability, they measure it for just one group of people, in the particular physical and social environment in which those people reside. If you change the environment, the heritable trait will usually change.

This point is important because scientists have found that, for most socially important traits, genetics does play a role. But studies that show how individuals differ genetically in a trait say nothing about whether genetics is the cause of racial differences in those traits. A trait can be highly heritable, and, at the same time, racial differences in that trait can be entirely non-genetic. Height is a classic example: It's highly heritable, and yet, over the past century, the relative average height of people in different countries has changed significantly. White American men were taller than Dutch men in the 19th century, but now the Dutch are about 5 cm taller than Americans. This shift is too fast to be entirely or even mostly genetic — it's largely due to an improvement in conditions in the Netherlands over the past century. (Though one study suggests that there might be some genetic contribution to these Dutch gains.)

4. Culture has enormous effects on social outcomes.

The influence of culture on social outcomes is not just a hypothetical — there is a great deal of evidence that culture has a large effect on many of the unequal social outcomes that some would like to ascribe to biological differences between races or sexes. Those who urge us not to deny that biology contributes to human nature have a point, but they often short-change the significance of what really makes the human species exceptional — our culture. While grasping at speculative biological explanations for why women are underrepresented at Google, or why so many African-Americans are incarcerated, these people ignore the obvious driving cultural factors that are backed by much more evidence than biological theories.

As many pointed out in response to James Damore's memo, women played foundational roles in computer science decades before men began to dominate the field. This clearly suggests that culture, not biology, is the big factor in the underrepresentation of women in tech. There is no question that African-Americans, and not just African-Americans living in poverty, experience a substantially different social environment than white Americans. In study after study, researchers have found clear evidence of racial and gender bias in everyday life, from education to employment to policing. It makes little sense to speculate about innate biological causes of racial and gender inequality, backed by little or no evidence, while downplaying the much more firmly grounded social explanations.

Our biology is, without question, an important part of who we are. But biology can't explain or excuse why women and non-whites are treated unequally in our society.

This story originally appeared as 4 good reasons you should be skeptical of the claim that biology explains inequality on Pacific Standard, an editorial partner site. Subscribe to the magazine's newsletter and follow Pacific Standard on Twitter to support journalism in the public interest.