Jeff Sessions says it’s “very painful” to be called a racist. That’s beside the point.

There was a very telling exchange about race at the Senate hearings for Alabama Sen. Jeff Sessions’s nomination for attorney general.

On Tuesday, South Carolina Sen. Lindsey Graham, a fellow Southern Republican, asked Sessions, “I am from South Carolina so I know what it means to be accused of being a conservative from the South — meaning a racist or a bigot. How does that make you feel?” Sessions responded that such accusations were “very painful.”

This is all too typical of discussions about race in America. Instead of considering what could drive someone to call another person racist, the issue quickly turns to just how unfair it feels to be called racist. The underlying feelings are never analyzed — just the perceptions of a word (“racist”) that’s now treated almost as a slur on its own.

This is part of a well-documented phenomenon called “white fragility.” The problem is people are often so sensitive to any accusations of racial bias that they may not have the ability to look beyond how they feel about being called a racist and evaluate what might merit such accusations — such as, in Sessions’s case, his history opposing civil and voting rights protections. And this is a particularly worrying trait for the person who may soon head the US Department of Justice, charged with enforcing laws that protect minority Americans’ civil and voting rights.

Over the past several decades, explicit racism has been socially stigmatized in America. As a result, being called a racist is supposed to impose a huge social burden on anyone, particularly on white people who may feel some sense of guilt for centuries of laws that imposed white supremacy through slavery, segregation, and voter suppression.

But this also means that when white people are called racist, even if it’s justified, they tend to become defensive — even hostile. Robin DiAngelo, who studies race at Westfield State University, described this phenomenon as “white fragility” in a groundbreaking 2011 paper:

White people in North America live in a social environment that protects and insulates them from race-based stress. This insulated environment of racial protection builds white expectations for racial comfort while at the same time lowering the ability to tolerate racial stress, leading to what I refer to as White Fragility. White Fragility is a state in which even a minimum amount of racial stress becomes intolerable, triggering a range of defensive moves. These moves include the outward display of emotions such as anger, fear, and guilt, and behaviors such as argumentation, silence, and leaving the stress-inducing situation. These behaviors, in turn, function to reinstate white racial equilibrium.

DiAngelo’s paper explained that white Americans have a range of “triggers” that can make them defensive about race, from suggestions that a person’s viewpoint is racialized to the rise of people of color into prominent leadership positions. All the triggers she listed were present in 2016 — through President Barack Obama’s elections, President-elect Donald Trump’s racist rhetoric, and Black Lives Matter protests against the dominance of white privilege.

Consider how often throughout the 2016 election people would respond to even the slightest suggestion of racism, whether in media or everyday life, with immediate vitriol, disdain, or dismissal. This, DiAngelo argued, is a defense mechanism to confronting questions about privilege. And it makes it difficult to have a reasonable conversation about race.

DiAngelo offered a telling example, from an anti-racism training session she facilitated:

One of the white participants left the session and went back to her desk, upset at receiving (what appeared to the training team as) sensitive and diplomatic feedback on how some of her statements had impacted several people of color in the room. At break, several other white participants approached us (the trainers) and reported that they had talked to the woman at her desk, and she was very upset that her statements had been challenged. They wanted to alert us to the fact that she literally “might be having a heart-attack.” Upon questioning from us, they clarified that they meant this literally. These co-workers were sincere in their fear that the young woman might actually physically die as a result of the feedback. Of course, when news of the woman’s potentially fatal condition reached the rest of the participant group, all attention was immediately focused back onto her and away from the impact she had had on the people of color.

This illustrates just how defensive people can get in the face of accusations of racism: Not only did the woman who faced the criticisms genuinely feel like she was having a heart attack, but the white people around her believed it was totally possible she was. This is the reality of trying to have a conversation about race in America.

So when someone hears accusations of racism, he may not hear what people are saying and why they may feel that way. Instead, he may feel insulted, offended, or defamed. This is what Sessions and Graham suggested during their exchange on Tuesday.

Maybe Sessions genuinely believes he’s not at all racist and that he truly views all people of all races as equal. So this makes an accusation of racism against him feel offensive and unwarranted, given what he thinks of himself. And we have no way of verifying that — only he knows what’s in his heart and mind.

But the issue is not about what’s in Sessions’s heart; it’s about what he’s done. And his record on this is perfectly clear: Time and time again, Sessions has backed policies that disproportionately hurt minority Americans and opposed policies that would help.

For one, Sessions has been an ardent opponent of voting rights protections. He previously said that the Voting Rights Act, which allowed the federal government to oversee elections in states with histories of racism, is a “piece of intrusive legislation.” And when the US Supreme Court struck down parts of the law, Sessions argued that Shelby County, Alabama, where the Supreme Court challenge came from, “has never had a history of denying voters and certainly not now” — even though the county, like much of Alabama, has a long, long history of discrimination of all kinds.

And as a federal prosecutor, Sessions went after activists who signed up black voters. In the 1980s, for example, he prosecuted Albert Turner for alleged voter fraud after Turner helped black voters register in Alabama — earning the nickname “Mr. Voter Registration.” The charges fell flat, with a jury deliberating for less than three hours before finding Turner not guilty on all counts of mail fraud, altering absentee ballots, and conspiracy to vote more than once. This moment is very telling: These kinds of court challenges would be tried time and time again by conservative lawmakers and prosecutors, even in 2016, to stop black voter registration efforts.

Then there’s Sessions’s history of more explicitly racist comments. When he was nominated for a federal judgeship in 1986, several witnesses said that Sessions had repeatedly made racist remarks, including a comment that the Ku Klux Klan “was okay until I found out they smoked pot.” (Sessions has now denied some of the comments, while claiming that some of the other remarks were meant to be jokes and were taken out of context.)

When you put this and Sessions’s other remarks and actions together, it’s easy to see why people are worried that Sessions may act in a racist way as US attorney general.

But instead of getting into the substance of any of the accusations, Graham’s line of questioning assumed they were baseless — that they were simply ridiculous insults.

Other senators have been trying to hold Sessions accountable for his record. And Sessions, for his part, flatly denied that he’s racist: “I did not harbor the kind of animosities and race-based discrimination ideas that I was accused of.”

Maybe Sessions feels that way, but his record tells civil rights advocates a different story — and many of them, from the NAACP to ACLU, have strongly condemned his nomination. And although it may be hard to hear, it’s something that Sessions will need to confront if he’s to head the agency in charge of protecting people’s civil and voting rights.