Cross Validated
2020-11-11 05:16 UTC
By semidevil
AI-113-20201111-social-media-b6871e58
conceptually, how to use NLP to predict a numeric output
suppose I'm trying to use medical notes to predict the cost of medical service. For example, a patient will call in, tell the operator how they feel, their diagnosis, etc etc, and the operator will take down all the notes. Over time, as the patient visits the doctor, gets medication, treatment, they would incur the final cost of the service. My job is to predict that final cost (as close as possible) based on the initial notes. I'm familiar with how to do standard EDA on the text; word frequency, sentiment, n-grams etc etc, but I dont know how that translates to a numerical output. As an analogy, if this was a linear regression, it's intuitive to me in the sense of each variable having a beta and I can say "for each x1 increase, y will increase by whatever %..." But for text, how does it work? Should the approach to find keywords that result in a certain value? i,e, there should be a relationship between the word "cancer" and cost, etc? Trying to wrap my head around this and get a sense of where to start and what to be look for.
suppose I'm trying to use medical notes to predict the cost of medical service. For example, a patient will call in, tell the operator how they feel, their diagnosis, etc etc, and the operator will take down all the notes. Over time, as the patient visits the doctor, gets medication, treatment, they would incur the final cost of the service. My job is to predict that final cost (as close as possible) based on the initial notes. I'm familiar with how to do standard EDA on the text; word frequency, sentiment, n-grams etc etc, but I dont know how that translates to a numerical output. As an analogy, if this was a linear regression, it's intuitive to me in the sense of each variable having a beta and I can say "for each x1 increase, y will increase by whatever %..." But for text, how does it work? Should the approach to find keywords that result in a certain value? i,e, there should be a relationship between the word "cancer" and cost, etc? Trying to wrap my head around this and get a sense of where to start and what to be look for.
Full article content could not be extracted automatically. Read the original below.
Source:
Cross Validated
· stats.stackexchange.com