4.3. EXPLAINABLE FAKE NEWS DETECTION 69
4.3 EXPLAINABLE FAKE NEWS DETECTION
In recent years, computational detection of fake news has been producing some promising early
results. However, there is a critical missing piece of the study, the explainability of such detection,
i.e., why a particular piece of news is detected as fake. Here, we introduce two representative
approaches based on web articles and user comments.
4.3.1 A WEB EVIDENCE-AWARE APPROACH
Web evidence is important to provide additional auxiliary information to predict the credibility
of online misinformation and fake news. Existing methods for fake news detection focus on ex-
ploring effective features from different sources such as the news content or social media signals
to improve fake news detection performance. However, these approaches also do not offer any
explanation of their verdicts. In the real world, external evidence or counter-evidence from the
Web can serve as a base to mine user-unrepeatable explanations.
Given a set of n news claims A D fa
1
; ; a
N
g with their corresponding sources P D
fp
1
; ; p
N
g, and each news claim a
i
is reported by a set of L articles W
i
D fe
i;1
; ; e
i;L
g,
where i 2 Œ1; N , from sources WP
i
D fep
i;1
; ; ep
i;L
g. Each tuple .a
i
; p
i
; e
ij
; ep
ij
/ forms a
training instance. e goal is to predict the label for each news claim as fake or real, with user-
comprehensible explanations for the prediction results (see Figure 4.9).
False
True
√
𝖷
Textual
Claim
DeClarE*
Evidence
World Wide
Web
Figure 4.9: e illustration of fake news detection with evidence-aware explanations.
e framework DeClarE[110] (see Figure 4.10), debunking claims with interpretable ev-
idence, mainly consists of the following components: (1) learning Claims and Article Represen-
tations; (2) claim Specific Attention; and (3) claim Credibility Prediction.
70 4. CHALLENGING PROBLEMS OF FAKE NEWS DETECTION
Claim Word
Embeddings
Claim Source
Embedding
Attention
Weights
Inner
Product
Concatenate
Softmax
Dense
Layer
Dense
Layer
Dense
Layer
Softmax/
Linear
Credibility
Score
Bidirectional
LSTM
Article Word
Embeddings
Article Source
Embedding
Concatenate
Features
avg
avgx
x
x
Figure 4.10: e framework of fake news detection with evidence-aware explanations. Based
on [110].
Learning Claims and Article Representations
e claim a
i
of length l is represented as Œw
1
; w
2
; ; w
l
where w
l
is the word embedding
vectors for the l-th word. e source of the claim and articles are represented by embed-
ding vectors with the same dimensions. A report article from the Web e
i;j
is represented by
Œw
i;j;1
; w
i;j;2
; ; w
i;j;k
, where w
i;j;k
is the word embedding vector of the k-th word in the
article. To obtain the representation of an article e
ij
, we can use the bidirectional LSTM net-
works, as we introduced in Section 2.1.3, to learn the neural textual representations. Specifically,
given an input word embedding of token w
k
, an LSTM cell performs nonlinear transformations
to generate a hidden state h
k
for timestamp k, and the last hidden state can be regard as the
representation. Since bidirectional LSTM is adopted, so the final out representation is the con-
catenation of the output of forward LSTM and backward LSTM, i.e., h D Œ
!
h ;
h .
Claim-Specific Attention
To consider the relevance of an article with respect to the claim, we can use attention mechanism
to help the model focus on salient words in the article. By adding attention mechanism, it also
helps make the model transparent and interpretable. First, the overall representation of the input
claim is generated by taking the average of the word embeddings of all words:
Na D
1
l
X
l
w
l
(4.20)
4.3. EXPLAINABLE FAKE NEWS DETECTION 71
and then the overall representation of this claim a is concatenated with each article as follows:
O
a
k
D e
k
˚ Na (4.21)
the claim-specific representation of each article is transformed through a fully connected layer:
a
0
k
D f .W
a
Oa
k
C b
a
/; (4.22)
where W
a
and b
a
are the corresponding weight matrix and bias terms, and f is the activation
function. Following this step, a softmax activation can be used to calculate an attention score ˛
k
for each word in the article capturing the relevance to the claim context:
˛
k
D
exp.a
0
k
/
P
k
exp.a
0
k
/
: (4.23)
Now that we have the article representation < h
k
>, and their relevance to the claim given
by < ˛
k
>, we can combine then to further predict the news claims’ credibility. e weighted
average of the hidden state representations for all articles can be calculated as follows:
r D
1
k
X
k
˛
k
h
k
: (4.24)
At last, the article representation r is combined with the claim source embedding (ep) and article
source embedding (p) simultaneously through a fully connected layer,
l D relu.W
c
.r ˚ ep ˚ p/ C b
c
/: (4.25)
Claim Credibility Prediction
e credibility score for each article y is predicted by taking the aforementioned representation
into a softmax layer:
e D softmax.l/: (4.26)
erefore, once we have the per-article credibility scores, we can take the average of these scores
to generate the overall credibility score for the news claim:
y D
1
L
X
j
e
j
: (4.27)
4.3.2 A SOCIAL CONTEXT-AWARE APPROACH
One way is to derive explanation from the perspectives of news contents and user comments
(see Figure 4.11) [132]. First, news contents may contain information that is verifiably false. For
example, journalists manually check the claims in news articles on fact-checking websites such
72 4. CHALLENGING PROBLEMS OF FAKE NEWS DETECTION
as PolitiFact,
1
which is usually labor intensive and time consuming. Researchers also attempt
to use external sources to fact-check the claims in news articles to decide and explain whether
a news piece is fake or not [29], which may not be able to check newly emerging events (that
has not been fact-checked). Second, user comments have rich information from the crowd on
social media, including opinions, stances, and sentiment, that are useful to detect fake news.
For example, researchers propose to use social features to select important comments to predict
fake news pieces [48]. Moreover, news contents and user comments inherently are related each
other and can provide important cues to explain why a given news article is fake or not. For
example, in Figure 4.11, we can see users discuss different aspects of the news in comments such
as “St. Nicholas was white? Really??Lol,” which directly responds to the claims in the
news content “The Holy Book always said Santa Claus was white.”
Figure 4.11: A piece of fake news on PolitiFact, and the user comments on social media. Some
explainable comments are directly related to the sentences in news contents.
Let a be a news article, consisting of L sentences fs
i
g
L
iD1
. Each sentence s
i
D
fw
i
1
; ; w
i
M
i
g contains M
i
words. Let C D fc
1
; c
2
; :::; c
T
g be a set of T comments related to
the news a, where each comment c
j
D fw
j
1
; ; w
j
Q
j
g contains Q
j
words. Similar to previous
research [59, 136], we treat fake news detection problem as the binary classification problem,
i.e., each news article can be true (y D 1) or fake (y D 0). At the same time, we aim to learn
a rank list RS from all sentences in fs
i
g
L
iD1
, and a rank list RC from all comments in fc
j
g
T
j D1
,
according to the degree of explainability, where RS
k
(RC
k
) denotes the k
th
most explainable sen-
tence (comment). e explainability of sentences in news contents represent the degree of how
check-worthy they are, while the explainability of comments denote the degree of how much
users believe if news is fake or real, closely related to the major claims in news.
1
https://www.politifact.com/
4.3. EXPLAINABLE FAKE NEWS DETECTION 73
We present the details of the framework for explainability fake news detection, named
as dEFEND (neural Explainable FakE News Detection). It consists of four major components
(see Figure 4.12): (1) a news content encoder (including word encoder and sentence encoder)
component; (2) a user comment encoder component; (3) a sentence-comment co-attention com-
ponent; and (4) a fake news prediction component.
s
S C
s
s
ŷ: Softmax
[ŝ,ĉ]: Concate
ĉ: Product
a
c
: Softmax
H
c
: Sum
L
T
: Product
ŝ: Product
a
s
: Softmax
H
s
: Sum
L: Product
α
α
α
M
β
β
β
M
h
h
h
N
v
o
w
o
c
v
v
N
h
h
h
N
h
h
h
M
w
w
w
M
h
h
h
M
c
c
c
T
······
······
······
······
······
······
······
h
h
h
Q
w
w
w
Q
h
h
h
Q
······
······
······
······
······
Sentence-Comment
Co-attention
Fake News
Prediction
Sentence Encoder
Comments Encoder
Word Encoder
Figure 4.12: e framework dEFEND consists of four components: (1) a news content (in-
cluding word-level and sentence-level) encoder; (2) a user comment encoder; (3) a sentence-
comment co-attention component; and (4) a fake news prediction component. Based on [132].
First, the news content encoder component describes the modeling from the news lin-
guistic features to latent feature space through a hierarchical word- and sentence-level encod-
74 4. CHALLENGING PROBLEMS OF FAKE NEWS DETECTION
ing; next, the user comment encoder component illustrates the comment latent feature extrac-
tion through word-level attention networks; then, the sentence-comment co-attention compo-
nent models the mutual influences between the news sentences and user comments for learning
feature representations, and the explainability degree of sentences and comments are learned
through the attention weights within co-attention learning; finally, the fake news prediction
component shows the process of concatenating news content and user comment features for
fake news classification.
News Contents Encoding As fake news pieces are intentionally created to spread inaccurate
information rather than to report objective claims, they often have opinionated and sensational
language styles, which have the potential to help detect fake news. In addition, a news document
contains linguistic cues with different levels such as word-level and sentence-level, which provide
different degrees of importance for the explainability of why the news is fake. For example,
in a fake news claim “Pence: Michelle Obama is the most vulgar first lady we've
ever had,” the word “vulgar” contributes more signals to decide whether the news claim is
fake rather than other words in the sentence.
Recently, researchers find that hierarchical attention neural networks [177] are very prac-
tical and useful to learn document representations [24] with highlighting important words or
sentences for classification. It adopts a hierarchical neural network to model word-level and
sentence-level representations through self-attention mechanisms. Inspired by [24], we learn
the news content representations through a hierarchical structure. Specifically, we first learn the
sentence vectors by using the word encoder with attention and then learn the sentence repre-
sentations through sentence encoder component.
Word Encoder We learn the sentence representation via a RNN based word encoder. Al-
though in theory, RNN is able to capture long-term dependency, in practice, the old mem-
ory will fade away as the sequence becomes longer. To making it easier for RNNs to capture
long-term dependencies, GRU [27] are designed in a manner to have more persistent memory.
Similar to [177], we adopt GRU to encode the word sequence.
To further capture the contextual information of annotations, we use bidirectional
GRU [8] to model word sequences from both directions of words. e bidirectional GRU con-
tains the forward GRU
!
f which reads sentence s
i
from word w
i1
to w
iM
i
and a backward GRU
f which reads sentence s
i
from word w
iM
i
to w
i1
:
!
h
it
D
!
GRU.w
it
/; t 2 f1; : : : ; M
i
g
h
it
D
GRU.w
it
/; t 2 fM
i
; : : : ; 1g:
(4.28)
We then obtain an annotation of word w
it
by concatenating the forward hidden state
!
h
it
and
backward hidden state
h
it
, i.e., h
it
D Œ
!
h
it
;
h
it
, which contains the information of the whole sen-
tence centered around w
it
.
4.3. EXPLAINABLE FAKE NEWS DETECTION 75
Note that not all words contribute equally to the representation of the sentence meaning.
erefore, we introduce an attention mechanism to learn the weights to measure the importance
of each word, and the sentence vector v
i
2 R
2d 1
is computed as follows:
v
i
D
M
i
X
tD1
˛
it
h
it
; (4.29)
where ˛
it
measures the importance of t
th
word for the sentence s
i
, and ˛
it
is calculated as follows:
o
it
D tanh.W
w
h
it
C b
w
/
˛
it
D
exp.o
it
o
T
w
/
P
M
i
kD1
exp.o
ik
o
T
w
/
;
(4.30)
where o
it
is a hidden representation of h
it
obtained by feeding the hidden state h
it
to a fully
embedding layer, and o
w
is the weight parameter that represents the world-level context vector.
Sentence Encoder Similar to word encoder, we utilize RNNs with GRU units to encode
each sentence in news articles. rough the sentence encoder, we can capture the context infor-
mation in the sentence-level to learn the sentence representations h
i
from the learned sentence
vector v
i
. Specifically, we can use the bidirectional GRU to encode the sentences as follows:
!
h
i
D
!
GRU.v
i
/; i 2 f1; : : : ; Lg
h
i
D
GRU.v
i
/; i 2 fL; : : : ; 1g:
(4.31)
We then obtain an annotation of sentence s
i
2 R
2d 1
by concatenating the forward hidden state
!
h
i
and backward hidden state
h
i
, i.e., s
i
D Œ
!
h
i
;
h
i
, which captures the context from neighbor
sentences around sentence s
i
.
User Comments Encoding People express their emotions or opinions toward fake news
through social media posts such as comments, such as skeptical opinions, sensational reactions,
etc. ese textual information has been shown to be related to the content of original news
pieces. us, comments may contain useful semantic information that has the potential to help
fake news detection. Next, we demonstrate how to encode the comments to learn the latent rep-
resentations. e comments extracted from social media are usually short text, so we use RNNs
to encode the word sequence in comments directly to learn the latent representations of com-
ments. Similar to the word encoder, we adopt bidirectional GRU to model the word sequences
in comments. Specifically, given a comment c
j
with words w
jt
; t 2 f1; ; Q
j
g, we first map
each word w
jt
into the word vector w
jt
2 R
d
with an embedding matrix. en, we can obtain
the feed forward hidden states
!
h
jt
and backward hidden states
h
jt
as follows:
!
h
jt
D
!
GRU.w
jt
/; t 2 f1; : : : ; Q
j
g
h
jt
D
GRU.w
jt
/; t 2 fQ
j
; : : : ; 1g:
(4.32)
76 4. CHALLENGING PROBLEMS OF FAKE NEWS DETECTION
We further obtain the annotation of word w
jt
by concatenating
!
h
jt
and
h
jt
, i.e., h
jt
D Œ
!
h
jt
;
h
jt
.
We also introduce the attention mechanism to learn the weights to measure the importance of
each word, and the comment vector c
j
2 R
2d
is computed as follows:
c
j
D
Q
j
X
tD1
ˇ
jt
h
jt
; (4.33)
where ˇ
jt
measures the importance of t
th
word for the comment c
j
, and ˇ
jt
is calculated as
follows,
o
jt
D tanh.W
c
h
jt
C b
c
/
ˇ
jt
D
exp.o
jt
o
T
c
/
P
Q
j
kD1
exp.o
j
k
o
T
c
/
;
(4.34)
where o
jt
is a hidden representation of h
jt
obtained by feeding the hidden state h
jt
to a fully
embedding layer, and u
c
is the weight parameter.
Sentence-Comment Co-attention We observe that not all sentences in news contents
are fake, and in fact, many sentences are true but only for supporting wrong claim sen-
tences [40]. us, news sentences may not be equally important in determining and explain-
ing whether a piece of news is fake or not. For example, the sentence “Michelle Obama is so
vulgar she's not only being vocal..” is strongly related to the major fake claim “Pence:
Michelle Obama Is The Most Vulgar First Lady We've Ever Had,” while “The First
Lady denounced the Republican presidential nominee” is a sentence that expresses
some fact and is less helpful in detecting and explaining whether the news is fake.
Similarly, user comments may contain relevant information about the important aspects
that explain why a piece of news is fake, while they may also be less informative and noisy. For ex-
ample, a comment “Where did Pence say this? I saw him on CBS this morning and
he didn't say these things..” is more explainable and useful to detect the fake news, than
other comments such as “Pence is absolutely right.”
us, we aim to select some news sentences and user comments that can explain why a
piece of news is fake. As they provide a good explanation, they should also be helpful in detecting
fake news. is suggests us to design attention mechanisms to give high weights of representa-
tions of news sentences and comments that are beneficial to fake news detection. Specifically,
we use sentence-comment co-attention because it can capture the semantic affinity of sentences
and comments and further help learn the attention weights of sentences and comments simul-
taneously.
We can construct the feature matrix of news sentences S D Œs
1
I ; s
L
2 R
2d L
and the
feature map of user comments C D fc
1
; ; c
T
g 2 R
2d T
, the co-attention attends to the sen-
tences and comments simultaneously. Similar to [83, 172], we first compute the affinity matrix
F 2 R
T L
as follows:
F D tanh.C
T
W
l
S/; (4.35)
4.3. EXPLAINABLE FAKE NEWS DETECTION 77
where W
l
2 R
2d 2d
is a weight matrix to be learned through the networks. Following the op-
timization strategy in [83], we can consider the affinity matrix as a feature and learn to predict
sentence and comment attention maps as follows:
H
s
D tanh
.
W
s
S C .W
c
C/F
/
H
c
D tanh
W
c
C C .W
s
S/F
T
;
(4.36)
where W
s
; W
c
2 R
k2d
are the weight parameters. e attention weights of sentences and com-
ments are calculated as follows:
a
s
D softmax
w
T
hs
H
s
a
c
D softmax
w
T
hc
H
c
;
(4.37)
where a
s
2 R
1N
and a
c
2 R
1T
are the attention probabilities of each sentence s
i
and comment
c
j
, respectively. w
hs
; w
hc
2 R
1k
are the weight parameters. e affinity matrix F transforms
user comment attention space to news sentence attention space, and vice versa for F
T
. Based on
the above attention weights, the comment and sentence attention vectors are calculated as the
weighted sum of the comment features and sentence features, i.e.,
Os D
L
X
iD1
a
s
i
s
i
; Oc D
T
X
j D1
a
c
j
c
j
; (4.38)
where Os 2 R
12d
and Oc 2 R
12d
are the learned features for news sentences and user comments
through co-attention.
Explainable Detection of Fake News We have introduced how we can encode news con-
tents by modeling the hierarchical structure from word level and sentence level, how we encode
comments by word-level attention networks, and the component to model co-attention to learn
sentences and comments representations. We further integrate these components together and
predict fake news with the following objective:
Oy D softmax.ŒOs; OcW
f
C b
f
/; (4.39)
where Oy D ŒOy
0
; Oy
1
is the predicted probability vector with Oy
0
and Oy
1
indicate the predicted prob-
ability of label being 0 (real news) and 1 (fake news) respectively. y 2 f0; 1g denotes the ground
truth label of news. ŒOs; Oc means the concatenation of learned features for news sentences and
user comments. b
f
2 R
12
is the bias term. us, for each news piece, the goal is to minimize
the cross-entropy loss function as follows:
L./ D y log.Oy
1
/ .1 y/ log.1 Oy
0
/;
(4.40)
where denotes the parameters of the network.
