Joint Optimization of Lithology and Petrophysical Parameters in Athabasca Oil Sands Using Self-Attention Mechanism

Precisely determining lithology and petrophysical parameters, including core-calibrated porosity and water saturation, is crucial for reservoir characterization. The traditional method of manually interpreting well-log data is not only time-consuming but also prone to human errors. To address these challenges in identifying lithology and estimating petrophysical parameters in Athabasca Oil Sands region, this study introduces a novel solution using the AutoRegressive Vision Transformer (ARViT) model for accurate prediction. ARViT improves upon the ViT framework by integrating sequential dependencies into its output. The self-attention mechanism and auto-regression are key features that enable ARViT to systematically process data, capturing detailed spatial dependencies in well-log data. This empowers the model to discern subtle spatial and temporal relationships among various geophysical measurements. In essence, ARViT's incorporation of sequential information through its auto-regression mechanism on top of ViT enhances its ability to comprehensively model complex relationships within well-log data. In this study, a multitask learning approach is embraced to enhance the model's interpretability and efficiency. This methodology involves optimizing the model's performance across multiple tasks simultaneously. By doing so, the model gains a broader understanding of diverse tasks and benefits from shared knowledge and features across these tasks. This collaborative optimization contributes to a more robust and versatile model, ultimately improving its overall perflormance and interpretability. To evaluate the effectiveness of the ARViT model, we conducted a comprehensive series of experiments and comparative analyses, contrasting its performance with conventional artificial neural networks (ANN), Long Short-Term Memory (LSTM), and ViT models. Furthermore, to illustrate the versatility of ARViT, we apply Low-Rank Adaptation (LoRA) to a different, smaller dataset of well-log, showcasing its ability to adapt effectively to various geological contexts. LoRA becomes particularly crucial in this context, as it not only enhances the model's adaptability but also plays a vital role in reducing the number of trainable parameters. This reduction not only contributes to computational efficiency but is essential for preventing overfitting and ensuring optimal performance across different datasets. Our findings demonstrate the consistent superiority of ARViT over ANN, LSTM, and ViT in accurately estimating lithological and petrophysical parameters. This is highlighted by ARViT's remarkable Lithological Accuracy of 96.51%, surpassing the baseline ANN's 73.18%, LSTM's 89.80%, and ViT's 93.23%. The substantial reduction in Mean Squared Error (MSE) for porosity, decreasing from 0.0007 (ANN) to 0.0004 (ARViT), and water saturation, decreasing from 0.022 (ANN) to 0.005 (ARViT), further emphasizes ARViT's exceptional performance in providing precise and reliable predictions across various metrics. The application of LoRA yields notable enhancements in ARViT's performance metrics. Specifically, in terms of Lithology Accuracy, ARViT-LoRA showcases a significant improvement, soaring from 88.74% (ARViT-Scratch) to an impressive 97.22%. Additionally, the implementation of LoRA resulted in a significant reduction of GPU consumption by 25%. While lithology prediction has been a well-explored field, ARViT distinguishes itself through its exclusive combination of features, encompassing a self-attention mechanism, auto-regressive nature, and multitask approach, coupled with effective fine-tuning using LoRA. This unique combination positions ARViT as a valuable tool for addressing intricate challenges of lithology prediction and petrophysical parameter estimation.