{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 0
},
"source": [
"# Neural Collaborative Filtering for Personalized Ranking\n",
"\n",
"This section moves beyond explicit feedback, introducing the neural collaborative filtering (NCF) framework for recommendation with implicit feedback. Implicit feedback is pervasive in recommender systems. Actions such as Clicks, buys, and watches are common implicit feedback which are easy to collect and indicative of users' preferences. The model we will introduce, titled NeuMF :cite:`He.Liao.Zhang.ea.2017`, short for neural matrix factorization, aims to address the personalized ranking task with implicit feedback. This model leverages the flexibility and non-linearity of neural networks to replace dot products of matrix factorization, aiming at enhancing the model expressiveness. In specific, this model is structured with two subnetworks including generalized matrix factorization (GMF) and MLP and models the interactions from two pathways instead of simple dot products. The outputs of these two networks are concatenated for the final prediction scores calculation. Unlike the rating prediction task in AutoRec, this model generates a ranked recommendation list to each user based on the implicit feedback. We will use the personalized ranking loss introduced in the last section to train this model.\n",
"\n",
"## The NeuMF model\n",
"\n",
"As aforementioned, NeuMF fuses two subnetworks. The GMF is a generic neural network version of matrix factorization where the input is the elementwise product of user and item latent factors. It consists of two neural layers:\n",
"\n",
"$$\n",
"\\mathbf{x} = \\mathbf{p}_u \\odot \\mathbf{q}_i \\\\\n",
"\\hat{y}_{ui} = \\alpha(\\mathbf{h}^\\top \\mathbf{x}),\n",
"$$\n",
"\n",
"where $\\odot$ denotes the Hadamard product of vectors. $\\mathbf{P} \\in \\mathbb{R}^{m \\times k}$ and $\\mathbf{Q} \\in \\mathbb{R}^{n \\times k}$ corespond to user and item latent matrix respectively. $\\mathbf{p}_u \\in \\mathbb{R}^{ k}$ is the $u^\\mathrm{th}$ row of $P$ and $\\mathbf{q}_i \\in \\mathbb{R}^{ k}$ is the $i^\\mathrm{th}$ row of $Q$. $\\alpha$ and $h$ denote the activation function and weight of the output layer. $\\hat{y}_{ui}$ is the prediction score of the user $u$ might give to the item $i$.\n",
"\n",
"Another component of this model is MLP. To enrich model flexibility, the MLP subnetwork does not share user and item embeddings with GMF. It uses the concatenation of user and item embeddings as input. With the complicated connections and nonlinear transformations, it is capable of estimating the intricate interactions between users and items. More precisely, the MLP subnetwork is defined as:\n",
"\n",
"$$\n",
"\\begin{aligned}\n",
"z^{(1)} &= \\phi_1(\\mathbf{U}_u, \\mathbf{V}_i) = \\left[ \\mathbf{U}_u, \\mathbf{V}_i \\right] \\\\\n",
"\\phi^{(2)}(z^{(1)}) &= \\alpha^1(\\mathbf{W}^{(2)} z^{(1)} + b^{(2)}) \\\\\n",
"&... \\\\\n",
"\\phi^{(L)}(z^{(L-1)}) &= \\alpha^L(\\mathbf{W}^{(L)} z^{(L-1)} + b^{(L)})) \\\\\n",
"\\hat{y}_{ui} &= \\alpha(\\mathbf{h}^\\top\\phi^L(z^{(L-1)}))\n",
"\\end{aligned}\n",
"$$\n",
"\n",
"where $\\mathbf{W}^*, \\mathbf{b}^*$ and $\\alpha^*$ denote the weight matrix, bias vector, and activation function. $\\phi^*$ denotes the function of the corresponding layer. $\\mathbf{z}^*$ denotes the output of corresponding layer.\n",
"\n",
"To fuse the results of GMF and MLP, instead of simple addition, NeuMF concatenates the second last layers of two subnetworks to create a feature vector which can be passed to the further layers. Afterwards, the ouputs are projected with matrix $\\mathbf{h}$ and a sigmoid activation function. The prediction layer is formulated as:\n",
"$$\n",
"\\hat{y}_{ui} = \\sigma(\\mathbf{h}^\\top[\\mathbf{x}, \\phi^L(z^{(L-1)})]).\n",
"$$\n",
"\n",
"The following figure illustrates the model architecture of NeuMF.\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "1"
},
"origin_pos": 1,
"tab": [
"mxnet"
]
},
"outputs": [],
"source": [
"import random\n",
"import mxnet as mx\n",
"from mxnet import autograd, gluon, np, npx\n",
"from mxnet.gluon import nn\n",
"from d2l import mxnet as d2l\n",
"\n",
"npx.set_np()"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 2
},
"source": [
"## Model Implementation\n",
"The following code implements the NeuMF model. It consists of a generalized matrix factorization model and an MLP with different user and item embedding vectors. The structure of the MLP is controlled with the parameter `nums_hiddens`. ReLU is used as the default activation function.\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "2"
},
"origin_pos": 3,
"tab": [
"mxnet"
]
},
"outputs": [],
"source": [
"class NeuMF(nn.Block):\n",
" def __init__(self, num_factors, num_users, num_items, nums_hiddens,\n",
" **kwargs):\n",
" super(NeuMF, self).__init__(**kwargs)\n",
" self.P = nn.Embedding(num_users, num_factors)\n",
" self.Q = nn.Embedding(num_items, num_factors)\n",
" self.U = nn.Embedding(num_users, num_factors)\n",
" self.V = nn.Embedding(num_items, num_factors)\n",
" self.mlp = nn.Sequential()\n",
" for num_hiddens in nums_hiddens:\n",
" self.mlp.add(nn.Dense(num_hiddens, activation='relu',\n",
" use_bias=True))\n",
" self.prediction_layer = nn.Dense(1, activation='sigmoid', use_bias=False)\n",
"\n",
" def forward(self, user_id, item_id):\n",
" p_mf = self.P(user_id)\n",
" q_mf = self.Q(item_id)\n",
" gmf = p_mf * q_mf\n",
" p_mlp = self.U(user_id)\n",
" q_mlp = self.V(item_id)\n",
" mlp = self.mlp(np.concatenate([p_mlp, q_mlp], axis=1))\n",
" con_res = np.concatenate([gmf, mlp], axis=1)\n",
" return self.prediction_layer(con_res)"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 4
},
"source": [
"## Customized Dataset with Negative Sampling\n",
"\n",
"For pairwise ranking loss, an important step is negative sampling. For each user, the items that a user has not interacted with are candidate items (unobserved entries). The following function takes users identity and candidate items as input, and samples negative items randomly for each user from the candidate set of that user. During the training stage, the model ensures that the items that a user likes to be ranked higher than items he dislikes or has not interacted with.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "3"
},
"origin_pos": 5,
"tab": [
"mxnet"
]
},
"outputs": [],
"source": [
"class PRDataset(gluon.data.Dataset):\n",
" def __init__(self, users, items, candidates, num_items):\n",
" self.users = users\n",
" self.items = items\n",
" self.cand = candidates\n",
" self.all = set([i for i in range(num_items)])\n",
"\n",
" def __len__(self):\n",
" return len(self.users)\n",
"\n",
" def __getitem__(self, idx):\n",
" neg_items = list(self.all - set(self.cand[int(self.users[idx])]))\n",
" indices = random.randint(0, len(neg_items) - 1)\n",
" return self.users[idx], self.items[idx], neg_items[indices]"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 6
},
"source": [
"## Evaluator\n",
"In this section, we adopt the splitting by time strategy to construct the training and test sets. Two evaluation measures including hit rate at given cutting off $\\ell$ ($\\text{Hit}@\\ell$) and area under the ROC curve (AUC) are used to assess the model effectiveness. Hit rate at given position $\\ell$ for each user indicates that whether the recommended item is included in the top $\\ell$ ranked list. The formal definition is as follows:\n",
"\n",
"$$\n",
"\\text{Hit}@\\ell = \\frac{1}{m} \\sum_{u \\in \\mathcal{U}} \\textbf{1}(rank_{u, g_u} <= \\ell),\n",
"$$\n",
"\n",
"where $\\textbf{1}$ denotes an indicator function that is equal to one if the ground truth item is ranked in the top $\\ell$ list, otherwise it is equal to zero. $rank_{u, g_u}$ denotes the ranking of the ground truth item $g_u$ of the user $u$ in the recommendation list (The ideal ranking is 1). $m$ is the number of users. $\\mathcal{U}$ is the user set.\n",
"\n",
"The definition of AUC is as follows:\n",
"\n",
"$$\n",
"\\text{AUC} = \\frac{1}{m} \\sum_{u \\in \\mathcal{U}} \\frac{1}{|\\mathcal{I} \\backslash S_u|} \\sum_{j \\in I \\backslash S_u} \\textbf{1}(rank_{u, g_u} < rank_{u, j}),\n",
"$$\n",
"\n",
"where $\\mathcal{I}$ is the item set. $S_u$ is the candidate items of user $u$. Note that many other evaluation protocols such as precision, recall and normalized discounted cumulative gain (NDCG) can also be used.\n",
"\n",
"The following function calculates the hit counts and AUC for each user.\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "4"
},
"origin_pos": 7,
"tab": [
"mxnet"
]
},
"outputs": [],
"source": [
"#@save\n",
"def hit_and_auc(rankedlist, test_matrix, k):\n",
" hits_k = [(idx, val) for idx, val in enumerate(rankedlist[:k])\n",
" if val in set(test_matrix)]\n",
" hits_all = [(idx, val) for idx, val in enumerate(rankedlist)\n",
" if val in set(test_matrix)]\n",
" max = len(rankedlist) - 1\n",
" auc = 1.0 * (max - hits_all[0][0]) / max if len(hits_all) > 0 else 0\n",
" return len(hits_k), auc"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 8
},
"source": [
"Then, the overall Hit rate and AUC are calculated as follows.\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "5"
},
"origin_pos": 9,
"tab": [
"mxnet"
]
},
"outputs": [],
"source": [
"#@save\n",
"def evaluate_ranking(net, test_input, seq, candidates, num_users, num_items,\n",
" devices):\n",
" ranked_list, ranked_items, hit_rate, auc = {}, {}, [], []\n",
" all_items = set([i for i in range(num_users)])\n",
" for u in range(num_users):\n",
" neg_items = list(all_items - set(candidates[int(u)]))\n",
" user_ids, item_ids, x, scores = [], [], [], []\n",
" [item_ids.append(i) for i in neg_items]\n",
" [user_ids.append(u) for _ in neg_items]\n",
" x.extend([np.array(user_ids)])\n",
" if seq is not None:\n",
" x.append(seq[user_ids, :])\n",
" x.extend([np.array(item_ids)])\n",
" test_data_iter = gluon.data.DataLoader(\n",
" gluon.data.ArrayDataset(*x), shuffle=False, last_batch=\"keep\",\n",
" batch_size=1024)\n",
" for index, values in enumerate(test_data_iter):\n",
" x = [gluon.utils.split_and_load(v, devices, even_split=False)\n",
" for v in values]\n",
" scores.extend([list(net(*t).asnumpy()) for t in zip(*x)])\n",
" scores = [item for sublist in scores for item in sublist]\n",
" item_scores = list(zip(item_ids, scores))\n",
" ranked_list[u] = sorted(item_scores, key=lambda t: t[1], reverse=True)\n",
" ranked_items[u] = [r[0] for r in ranked_list[u]]\n",
" temp = hit_and_auc(ranked_items[u], test_input[u], 50)\n",
" hit_rate.append(temp[0])\n",
" auc.append(temp[1])\n",
" return np.mean(np.array(hit_rate)), np.mean(np.array(auc))"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 10
},
"source": [
"## Training and Evaluating the Model\n",
"\n",
"The training function is defined below. We train the model in the pairwise manner.\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "6"
},
"origin_pos": 11,
"tab": [
"mxnet"
]
},
"outputs": [],
"source": [
"#@save\n",
"def train_ranking(net, train_iter, test_iter, loss, trainer, test_seq_iter,\n",
" num_users, num_items, num_epochs, devices, evaluator,\n",
" candidates, eval_step=1):\n",
" timer, hit_rate, auc = d2l.Timer(), 0, 0\n",
" animator = d2l.Animator(xlabel='epoch', xlim=[1, num_epochs], ylim=[0, 1],\n",
" legend=['test hit rate', 'test AUC'])\n",
" for epoch in range(num_epochs):\n",
" metric, l = d2l.Accumulator(3), 0.\n",
" for i, values in enumerate(train_iter):\n",
" input_data = []\n",
" for v in values:\n",
" input_data.append(gluon.utils.split_and_load(v, devices))\n",
" with autograd.record():\n",
" p_pos = [net(*t) for t in zip(*input_data[0:-1])]\n",
" p_neg = [net(*t) for t in zip(*input_data[0:-2],\n",
" input_data[-1])]\n",
" ls = [loss(p, n) for p, n in zip(p_pos, p_neg)]\n",
" [l.backward(retain_graph=False) for l in ls]\n",
" l += sum([l.asnumpy() for l in ls]).mean()/len(devices)\n",
" trainer.step(values[0].shape[0])\n",
" metric.add(l, values[0].shape[0], values[0].size)\n",
" timer.stop()\n",
" with autograd.predict_mode():\n",
" if (epoch + 1) % eval_step == 0:\n",
" hit_rate, auc = evaluator(net, test_iter, test_seq_iter,\n",
" candidates, num_users, num_items,\n",
" devices)\n",
" animator.add(epoch + 1, (hit_rate, auc))\n",
" print(f'train loss {metric[0] / metric[1]:.3f}, '\n",
" f'test hit rate {float(hit_rate):.3f}, test AUC {float(auc):.3f}')\n",
" print(f'{metric[2] * num_epochs / timer.sum():.1f} examples/sec '\n",
" f'on {str(devices)}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 12
},
"source": [
"Now, we can load the MovieLens 100k dataset and train the model. Since there are only ratings in the MovieLens dataset, with some losses of accuracy, we binarize these ratings to zeros and ones. If a user rated an item, we consider the implicit feedback as one, otherwise as zero. The action of rating an item can be treated as a form of providing implicit feedback. Here, we split the dataset in the `seq-aware` mode where users' latest interacted items are left out for test.\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "11"
},
"origin_pos": 13,
"tab": [
"mxnet"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading ../data/ml-100k.zip from https://files.grouplens.org/datasets/movielens/ml-100k.zip...\n"
]
}
],
"source": [
"batch_size = 1024\n",
"df, num_users, num_items = d2l.read_data_ml100k()\n",
"train_data, test_data = d2l.split_data_ml100k(df, num_users, num_items,\n",
" 'seq-aware')\n",
"users_train, items_train, ratings_train, candidates = d2l.load_data_ml100k(\n",
" train_data, num_users, num_items, feedback=\"implicit\")\n",
"users_test, items_test, ratings_test, test_iter = d2l.load_data_ml100k(\n",
" test_data, num_users, num_items, feedback=\"implicit\")\n",
"train_iter = gluon.data.DataLoader(\n",
" PRDataset(users_train, items_train, candidates, num_items ), batch_size,\n",
" True, last_batch=\"rollover\", num_workers=d2l.get_dataloader_workers())"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 14
},
"source": [
"We then create and initialize the model. we use a three-layer MLP with constant hidden size 10.\n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "8"
},
"origin_pos": 15,
"tab": [
"mxnet"
]
},
"outputs": [],
"source": [
"devices = d2l.try_all_gpus()\n",
"net = NeuMF(10, num_users, num_items, nums_hiddens=[10, 10, 10])\n",
"net.initialize(ctx=devices, force_reinit=True, init=mx.init.Normal(0.01))"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 16
},
"source": [
"The following code trains the model.\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"attributes": {
"classes": [],
"id": "",
"n": "12"
},
"origin_pos": 17,
"tab": [
"mxnet"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"train loss 16.982, test hit rate 0.075, test AUC 0.531\n",
"13.2 examples/sec on [gpu(0), gpu(1)]\n"
]
},
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"lr, num_epochs, wd, optimizer = 0.01, 10, 1e-5, 'adam'\n",
"loss = d2l.BPRLoss()\n",
"trainer = gluon.Trainer(net.collect_params(), optimizer,\n",
" {\"learning_rate\": lr, 'wd': wd})\n",
"train_ranking(net, train_iter, test_iter, loss, trainer, None, num_users,\n",
" num_items, num_epochs, devices, evaluate_ranking, candidates)"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 18
},
"source": [
"## Summary\n",
"\n",
"* Adding nonlinearity to matrix factorization model is beneficial for improving the model capability and effectiveness.\n",
"* NeuMF is a combination of matrix factorization and multilayer perceptron. The multilayer perceptron takes the concatenation of user and item embeddings as the input.\n",
"\n",
"## Exercises\n",
"\n",
"* Vary the size of latent factors. How the size of latent factors impact the model performance?\n",
"* Vary the architectures (e.g., number of layers, number of neurons of each layer) of the MLP to check the its impact on the performance.\n",
"* Try different optimizers, learning rate and weight decay rate.\n",
"* Try to use hinge loss defined in the last section to optimize this model.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"origin_pos": 19,
"tab": [
"mxnet"
]
},
"source": [
"[Discussions](https://discuss.d2l.ai/t/403)\n"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 4
}