메뉴
R text mining with tidytext, tm, quanteda. Use for tokenization, TF-IDF, document-term matrices.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
R language data analysis and visualization skill. Use when user asks to (1) run R scripts or code, (2) install/update R packages, (3) perform data analysis with R, (4) create visualizations with ggplot2/plotly, (5) statistical analysis, (6) data manipulation with tidyverse/dplyr/data.table. Triggers on keywords like "R语言", "R脚本", "ggplot", "tidyverse", "数据分析", "可视化".
R DALEX package for model explanations. Use for explaining complex machine learning models.
R iml package for interpretable ML. Use for model-agnostic interpretability methods.
R lime package for local explanations. Use for explaining individual predictions with local interpretable models.
R packages for ML interpretability. Use for explaining and interpreting machine learning models.
R vip package for variable importance. Use for computing and visualizing variable importance scores.
| name | r-nlp-text |
| description | R text mining with tidytext, tm, quanteda. Use for tokenization, TF-IDF, document-term matrices. |
Text processing and analysis.
library(tidytext)
library(dplyr)
# Tokenize
df %>% unnest_tokens(word, text)
# Remove stop words
df %>%
unnest_tokens(word, text) %>%
anti_join(stop_words)
# Word counts
df %>%
unnest_tokens(word, text) %>%
count(word, sort = TRUE)
# TF-IDF
df %>%
unnest_tokens(word, text) %>%
count(document, word) %>%
bind_tf_idf(word, document, n)
# N-grams
df %>% unnest_tokens(bigram, text, token = "ngrams", n = 2)
# Cast to DTM
dtm <- df %>%
unnest_tokens(word, text) %>%
count(document, word) %>%
cast_dtm(document, word, n)
library(quanteda)
# Create corpus
corpus <- corpus(texts)
# Tokenize
tokens <- tokens(corpus, remove_punct = TRUE, remove_numbers = TRUE)
tokens <- tokens_tolower(tokens)
tokens <- tokens_remove(tokens, stopwords("en"))
tokens <- tokens_wordstem(tokens)
# Document-feature matrix
dfm <- dfm(tokens)
dfm <- dfm_trim(dfm, min_termfreq = 5, min_docfreq = 2)
# TF-IDF weighting
dfm_tfidf <- dfm_tfidf(dfm)
# Top features
topfeatures(dfm, 20)
# Keyword in context
kwic(tokens, pattern = "economy", window = 5)
library(tm)
# Create corpus
corpus <- Corpus(VectorSource(texts))
# Preprocessing
corpus <- tm_map(corpus, content_transformer(tolower))
corpus <- tm_map(corpus, removePunctuation)
corpus <- tm_map(corpus, removeNumbers)
corpus <- tm_map(corpus, removeWords, stopwords("english"))
corpus <- tm_map(corpus, stemDocument)
corpus <- tm_map(corpus, stripWhitespace)
# Document-term matrix
dtm <- DocumentTermMatrix(corpus)
dtm <- removeSparseTerms(dtm, 0.99)
# Term-document matrix
tdm <- TermDocumentMatrix(corpus)
# Find frequent terms
findFreqTerms(dtm, lowfreq = 10)
# Find associations
findAssocs(dtm, "economy", corlimit = 0.3)
library(text2vec)
# Iterator
it <- itoken(texts, tokenizer = word_tokenizer, progressbar = FALSE)
# Vocabulary
vocab <- create_vocabulary(it)
vocab <- prune_vocabulary(vocab, term_count_min = 5)
# Vectorizer
vectorizer <- vocab_vectorizer(vocab)
# DTM
dtm <- create_dtm(it, vectorizer)
# TF-IDF
tfidf <- TfIdf$new()
dtm_tfidf <- fit_transform(dtm, tfidf)