{ "cells": [ { "cell_type": "markdown", "metadata": { "origin_pos": 0 }, "source": [ "# Sentiment Analysis and the Dataset\n", ":label:`sec_sentiment`\n", "\n", "\n", "With the proliferation of online social media\n", "and review platforms,\n", "a plethora of\n", "opinionated data\n", "have been logged,\n", "bearing great potential for\n", "supporting decision making processes.\n", "*Sentiment analysis*\n", "studies people's sentiments\n", "in their produced text,\n", "such as product reviews,\n", "blog comments,\n", "and\n", "forum discussions.\n", "It enjoys wide applications\n", "to fields as diverse as \n", "politics (e.g., analysis of public sentiments towards policies),\n", "finance (e.g., analysis of sentiments of the market),\n", "and \n", "marketing (e.g., product research and brand management).\n", "\n", "Since sentiments\n", "can be categorized\n", "as discrete polarities or scales (e.g., positive and negative),\n", "we can consider \n", "sentiment analysis \n", "as a text classification task,\n", "which transforms a varying-length text sequence\n", "into a fixed-length text category.\n", "In this chapter,\n", "we will use Stanford's [large movie review dataset](https://ai.stanford.edu/~amaas/data/sentiment/)\n", "for sentiment analysis. \n", "It consists of a training set and a testing set, \n", "either containing 25000 movie reviews downloaded from IMDb.\n", "In both datasets, \n", "there are equal number of \n", "\"positive\" and \"negative\" labels,\n", "indicating different sentiment polarities.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "origin_pos": 1, "tab": [ "mxnet" ] }, "outputs": [], "source": [ "import os\n", "from mxnet import np, npx\n", "from d2l import mxnet as d2l\n", "\n", "npx.set_np()" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 3 }, "source": [ "## Reading the Dataset\n", "\n", "First, download and extract this IMDb review dataset\n", "in the path `../data/aclImdb`.\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "origin_pos": 4, "tab": [ "mxnet" ] }, "outputs": [], "source": [ "#@save\n", "d2l.DATA_HUB['aclImdb'] = (\n", " 'http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz',\n", " '01ada507287d82875905620988597833ad4e0903')\n", "\n", "data_dir = d2l.download_extract('aclImdb', 'aclImdb')" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 5 }, "source": [ "Next, read the training and test datasets. Each example is a review and its label: 1 for \"positive\" and 0 for \"negative\".\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "origin_pos": 6, "tab": [ "mxnet" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# trainings: 25000\n", "label: 1 review: Henry Hathaway was daring, as well as enthusiastic, for his \n", "label: 1 review: An unassuming, subtle and lean film, \"The Man in the White S\n", "label: 1 review: Eddie Murphy really made me laugh my ass off on this HBO sta\n" ] } ], "source": [ "#@save\n", "def read_imdb(data_dir, is_train):\n", " \"\"\"Read the IMDb review dataset text sequences and labels.\"\"\"\n", " data, labels = [], []\n", " for label in ('pos', 'neg'):\n", " folder_name = os.path.join(data_dir, 'train' if is_train else 'test',\n", " label)\n", " for file in os.listdir(folder_name):\n", " with open(os.path.join(folder_name, file), 'rb') as f:\n", " review = f.read().decode('utf-8').replace('\\n', '')\n", " data.append(review)\n", " labels.append(1 if label == 'pos' else 0)\n", " return data, labels\n", "\n", "train_data = read_imdb(data_dir, is_train=True)\n", "print('# trainings:', len(train_data[0]))\n", "for x, y in zip(train_data[0][:3], train_data[1][:3]):\n", " print('label:', y, 'review:', x[0:60])" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 7 }, "source": [ "## Preprocessing the Dataset\n", "\n", "Treating each word as a token\n", "and filtering out words that appear less than 5 times,\n", "we create a vocabulary out of the training dataset.\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "origin_pos": 8, "tab": [ "mxnet" ] }, "outputs": [], "source": [ "train_tokens = d2l.tokenize(train_data[0], token='word')\n", "vocab = d2l.Vocab(train_tokens, min_freq=5, reserved_tokens=[''])" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 9 }, "source": [ "After tokenization,\n", "let us plot the histogram of\n", "review lengths in tokens.\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "origin_pos": 10, "tab": [ "mxnet" ] }, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", " \n", " 2022-03-24T10:58:16.311201\n", " image/svg+xml\n", " \n", " \n", " Matplotlib v3.5.1, https://matplotlib.org/\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n" ], "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "d2l.set_figsize()\n", "d2l.plt.xlabel('# tokens per review')\n", "d2l.plt.ylabel('count')\n", "d2l.plt.hist([len(line) for line in train_tokens], bins=range(0, 1000, 50));" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 11 }, "source": [ "As we expected,\n", "the reviews have varying lengths.\n", "To process\n", "a minibatch of such reviews at each time,\n", "we set the length of each review to 500 with truncation and padding,\n", "which is similar to \n", "the preprocessing step \n", "for the machine translation dataset\n", "in :numref:`sec_machine_translation`.\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "origin_pos": 12, "tab": [ "mxnet" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(25000, 500)\n" ] } ], "source": [ "num_steps = 500 # sequence length\n", "train_features = np.array([d2l.truncate_pad(\n", " vocab[line], num_steps, vocab['']) for line in train_tokens])\n", "print(train_features.shape)" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 13 }, "source": [ "## Creating Data Iterators\n", "\n", "Now we can create data iterators.\n", "At each iteration, a minibatch of examples are returned.\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "origin_pos": 14, "tab": [ "mxnet" ] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X: (64, 500) , y: (64,)\n", "# batches: 391\n" ] } ], "source": [ "train_iter = d2l.load_array((train_features, train_data[1]), 64)\n", "\n", "for X, y in train_iter:\n", " print('X:', X.shape, ', y:', y.shape)\n", " break\n", "print('# batches:', len(train_iter))" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 16 }, "source": [ "## Putting All Things Together\n", "\n", "Last, we wrap up the above steps into the `load_data_imdb` function.\n", "It returns training and test data iterators and the vocabulary of the IMDb review dataset.\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "origin_pos": 17, "tab": [ "mxnet" ] }, "outputs": [], "source": [ "#@save\n", "def load_data_imdb(batch_size, num_steps=500):\n", " \"\"\"Return data iterators and the vocabulary of the IMDb review dataset.\"\"\"\n", " data_dir = d2l.download_extract('aclImdb', 'aclImdb')\n", " train_data = read_imdb(data_dir, True)\n", " test_data = read_imdb(data_dir, False)\n", " train_tokens = d2l.tokenize(train_data[0], token='word')\n", " test_tokens = d2l.tokenize(test_data[0], token='word')\n", " vocab = d2l.Vocab(train_tokens, min_freq=5)\n", " train_features = np.array([d2l.truncate_pad(\n", " vocab[line], num_steps, vocab['']) for line in train_tokens])\n", " test_features = np.array([d2l.truncate_pad(\n", " vocab[line], num_steps, vocab['']) for line in test_tokens])\n", " train_iter = d2l.load_array((train_features, train_data[1]), batch_size)\n", " test_iter = d2l.load_array((test_features, test_data[1]), batch_size,\n", " is_train=False)\n", " return train_iter, test_iter, vocab" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 19 }, "source": [ "## Summary\n", "\n", "* Sentiment analysis studies people's sentiments in their produced text, which is considered as a text classification problem that transforms a varying-length text sequence\n", "into a fixed-length text category.\n", "* After preprocessing, we can load Stanford's large movie review dataset (IMDb review dataset) into data iterators with a vocabulary.\n", "\n", "\n", "## Exercises\n", "\n", "\n", "1. What hyperparameters in this section can we modify to accelerate training sentiment analysis models?\n", "1. Can you implement a function to load the dataset of [Amazon reviews](https://snap.stanford.edu/data/web-Amazon.html) into data iterators and labels for sentiment analysis?\n" ] }, { "cell_type": "markdown", "metadata": { "origin_pos": 20, "tab": [ "mxnet" ] }, "source": [ "[Discussions](https://discuss.d2l.ai/t/391)\n" ] } ], "metadata": { "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 4 }