PY
py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
'''
makes a sentence simple but removing punctuations and less meaningful words
this program processes the string using "nltk" python module to make data clear so that we can train any kind of language model through that data
for an example
input = Hello! solo learner, How are you
output : ['hello', 'solo', 'learner']
from this output we can train any Language model for an example ChatGPT
step :-
input a statement with punctuations and uneven text case example "HeLlo! WorLD hoW are YoU"
'''
#-------------importing modules---------------|
import nltk
import string
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
#---------------------------------------------|
Enter to Rename, Shift+Enter to Preview
OUTPUT
Run