In agriculture, our farmers currently lack efficient methods to classify and identify plant nutrient deficiencies or diseases, leading to ineffective remedial measures. However, the potential for early detection of plant nutrient deficiencies using colour gradients or leaf patterns, coupled with the advancements in Image Processing, Deep Learning (DL), and Artificial Intelligence (AI), offers a promising future. This paper presents deep learning-based methods to segregate nutrient deficiencies using leaf images, a potential game-changer for the industry. Our research has the potential to revolutionize plant nutrition management, inspiring a new wave of efficient and effective practices in agriculture. We analyzed the International Plant Nutrition Institute (IPNI) dataset and applied an image augmentation mechanism to enhance the training process with the available dataset. By utilizing Artificial Neural Network (ANN), Convolutional Neural Network (CNN), and learning algorithms, we minimised loss and improved accuracy. The model, incorporating over 1.5 million parameters, has shown promising results in predicting deficiencies accurately. We propose an SGD-based training mechanism to balance localization and classification tasks, a step towards a more efficient and effective approach to plant nutrition management.