Most physical activity (PA) assessment studies use wearable accelerometers attached to the hip. However, there is significant and recent interest in understanding the usefulness of wrist based accelerometer data collection due to ease of use and higher compliance. This paper develops machine learning methods for identifying activity types and computing energy expenditures. Our approach converts the raw time series data into intermediate variables or features using standard statistical methods as well as bag-of-words (BoW) approach. We tested this approach to assess type of physical activities as well as estimate the required corresponding energy expenditure. The method was evaluated on 17 participants. Each of the participants wore an Actigraph GT3X+ accelerometer on the right wrist and performed 33 activities of daily living. Energy expenditure was measured in parallel by a portable indirect calorimetry system. Our results show that the BoW approach resulted in a more accurate model for PA identification (F1-score = 0.88 and 0.91 for sedentary and locomotion detection, respectively), compared with standard statistical summaries. The BoW approach preserved additional details about the accelerometer data, which resulted in distinguishing different activities that belonged to the same higher-level category (e.g., distinguishing leisure walk from stair ascent where both PAs belong to the locomotion class) and consequently yielding an accurate energy expenditure estimation model for PAs (rMSE = 0.93 and R2 = 0.69).