extract token using R

Question

I am trying to extract a token from a file. The problem is that the lines get merged and extracts every word with token as prefix or postfix.

Answer 1

Just add collapse = FALSE in your unnest_tokens:

library(tidytext)
library(dplyr)

jobs %>% 
  unnest_tokens(ngram, POSITION, token = "ngrams", n = 2, collapse = FALSE)