java.lang.NullPointerException when iteration through viewentrycollection

[PatrickKwinten]

• Domino server Release 12.0.1FP1 HF72
• DDE Revision 20220404.1552-FP1 (Release 12.0.1FP1)
• ODA version 12.0.2.202211211802

here is an example of the error from log.nsf:

java.lang.NullPointerException
2023-05-26 09:44:29 HTTP JVM: at org.openntf.domino.impl.ViewEntryCollection.getCount(ViewEntryCollection.java:219)
2023-05-26 09:44:29 HTTP JVM: at org.openntf.domino.iterators.ViewEntryIterator.(ViewEntryIterator.java:57)
2023-05-26 09:44:29 HTTP JVM: at org.openntf.domino.impl.ViewEntryCollection.iterator(ViewEntryCollection.java:111)
2023-05-26 09:44:29 HTTP JVM: at org.acme.kkom.dao.AttachmentDominoDAO.updateAccess(AttachmentDominoDAO.java:510)

510 refers to the for loop statement (for (ViewEntry entry : entries) ) in:

View vw = db.getView(propDataSources.getProperty("vw_attach_unid"));
if (null != vw){
ViewEntryCollection entries = vw.createViewEntryCollection();
if (null != key){
entries = vw.getAllEntriesByKey(key, true);
}
else{
entries = vw.getAllEntries();
}
//ODA way:
for (ViewEntry entry : entries) {
if(entry.isValid()){
if(entry.isDocument()){
Document doc = entry.getDocument();
if(null != item){
doc.copyItem(item);
} else{
doc.removeItem(fieldName);
}
if(doc.save(true,false)){
vw.refresh();
}
}
}
}
}

[PatrickKwinten]

Hi, we reported a suppport case at HCL and according to their findings the OpenNTF code is not threadsafe. Here is a snippet of their report:

While review the logs its not HTTP task related issue however this issue with a multi-threaded application code and issue seems to happen in the openntf framework and custom code use.
I have get reviewed this logs from extended support team and they checked in in JAVA core file and found the deadlock is detected and uses of the custom code in openntf framework
Please find below the function in BOLD letters :

[Thursday 5:01 PM] Niels Groot
3XMTHREADINFO "Thread-10" J9VMThread:0x00000000008DD900, omrthread_t:0x00000198D63EA400, java/lang/Thread:0x00000000C0C1D510, state:B, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x22, isDaemon:false)
3XMTHREADINFO1 (native thread ID:0x1884, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000281)
3XMCPUTIME CPU usage total: 2256.734375000 secs, user: 2182.031250000 secs, system: 74.703125000 secs, current category="Application"
3XMTHREADBLOCK Blocked on: org/openntf/domino/logging/LogFilterHandler@0x00000000C0C354A0 Owned by: "Thread-8" (J9VMThread:0x00000000008DC000, java/lang/Thread:0x00000000C0C1D3B0)
3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at org/openntf/domino/logging/LogFilterHandler.publish(LogFilterHandler.java:299(Compiled Code))
5XESTACKTRACE (entered lock: org/openntf/domino/logging/LogFilterHandler@0x00000000C0C354A0, entry count: 1)
4XESTACKTRACE at java/util/logging/Logger.log(Logger.java:738(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.doLog(Logger.java:765(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.log(Logger.java:788(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.fine(Logger.java:1516(Compiled Code))
4XESTACKTRACE at org/openntf/domino/impl/Base.getDelegate(Base.java:475(Compiled Code))
4XESTACKTRACE at org/openntf/domino/impl/Session.getDatabase(Session.java:791(Compiled Code))
5XESTACKTRACE (entered lock: java/lang/Object@0x00000000C0C1C688, entry count: 1)
4XESTACKTRACE at org/openntf/domino/impl/Session.getDatabase(Session.java:827(Compiled Code))
the "entered lock" reflect a java lock, one uses these for example to make an application thread safe
LogFilterHandler.publish(LogFilterHandler.java:299 ---> that's opensource openntf
Refer to : - https://github.com/OpenNTF/org.openntf.domino/blob/develop/domino/core/src/main/java/org/openntf/domino/logging/LogFilterHandler.java
some info around synchronization in java: https://codegym.cc/groups/posts/377-top-50-job-interview-questions-and-answers-for-java-core-part-2
better (its just to a quick grasp on multi thread sync/locks that one needs when coding a multithreaded app)
https://www.codejava.net/java-core/concurrency/java-synchronization-tutorial-part-1-the-problems-of-unsynchronized-code
Java Synchronization Tutorial Part 1 - The Problems of Unsynchronized Code
To know how synchronization works in Java, you need to understand what kinds of problem your application may face without synchronized code

[PatrickKwinten]

so same issue on iterating through viewentrycollection as documentcollection

[PatrickKwinten]

Custom code used: -
3XMTHREADINFO "Thread-10" J9VMThread:0x00000000008DD900, omrthread_t:0x00000198D63EA400, java/lang/Thread:0x00000000C0C1D510, state:B, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x22, isDaemon:false)
3XMTHREADINFO1 (native thread ID:0x1884, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000281)
3XMCPUTIME CPU usage total: 2256.734375000 secs, user: 2182.031250000 secs, system: 74.703125000 secs, current category="Application"
3XMTHREADBLOCK Blocked on: org/openntf/domino/logging/LogFilterHandler@0x00000000C0C354A0 Owned by: "Thread-8" (J9VMThread:0x00000000008DC000, java/lang/Thread:0x00000000C0C1D3B0)
3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at org/openntf/domino/logging/LogFilterHandler.publish(LogFilterHandler.java:299(Compiled Code))
5XESTACKTRACE (entered lock: org/openntf/domino/logging/LogFilterHandler@0x00000000C0C354A0, entry count: 1)
4XESTACKTRACE at java/util/logging/Logger.log(Logger.java:738(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.doLog(Logger.java:765(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.log(Logger.java:788(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.fine(Logger.java:1516(Compiled Code))
4XESTACKTRACE at org/openntf/domino/impl/Base.getDelegate(Base.java:475(Compiled Code))
4XESTACKTRACE at org/openntf/domino/impl/Session.getDatabase(Session.java:791(Compiled Code))
5XESTACKTRACE (entered lock: java/lang/Object@0x00000000C0C1C688, entry count: 1)
4XESTACKTRACE at org/openntf/domino/impl/Session.getDatabase(Session.java:827(Compiled Code))
4XESTACKTRACE at org/acme/kkom/dao/MatterDominoDAO.checkLatestDecision(MatterDominoDAO.java:8923)

3XMTHREADINFO "Thread-8" J9VMThread:0x00000000008DC000, omrthread_t:0x00000198D63E9A70, java/lang/Thread:0x00000000C0C1D3B0, state:B, prio=5
3XMJAVALTHREAD (java/lang/Thread getId:0x20, isDaemon:false)
3XMTHREADINFO1 (native thread ID:0x1878, native priority:0x5, native policy:UNKNOWN, vmstate:B, vm thread flags:0x00000281)
3XMCPUTIME CPU usage total: 4679.453125000 secs, user: 4555.359375000 secs, system: 124.093750000 secs, current category="Application"
3XMTHREADBLOCK Blocked on: java/lang/Object@0x00000000C0C1C688 Owned by: "Thread-10" (J9VMThread:0x00000000008DD900, java/lang/Thread:0x00000000C0C1D510)
3XMHEAPALLOC Heap bytes allocated since last GC cycle=0 (0x0)
3XMTHREADINFO3 Java callstack:
4XESTACKTRACE at org/openntf/domino/impl/Session.getDatabase(Session.java:790(Compiled Code))
4XESTACKTRACE at org/openntf/domino/impl/Session.getDatabase(Session.java:827(Compiled Code))
4XESTACKTRACE at org/openntf/domino/impl/Session.getDatabase(Session.java:840)
4XESTACKTRACE at org/openntf/domino/logging/LogGeneratorOpenLog$OL_Writer.getEmptyDocument(LogGeneratorOpenLog.java:216)
4XESTACKTRACE at org/openntf/domino/logging/LogGeneratorOpenLog$OL_Writer.writeLogRecToDB(LogGeneratorOpenLog.java:227)
4XESTACKTRACE at org/openntf/domino/logging/LogGeneratorOpenLog.log(LogGeneratorOpenLog.java:150)
5XESTACKTRACE (entered lock: org/openntf/domino/logging/LogGeneratorOpenLog@0x00000000C0C36D68, entry count: 1)
4XESTACKTRACE at org/openntf/domino/logging/LogHandlerOpenLog.publish(LogHandlerOpenLog.java:174)
5XESTACKTRACE (entered lock: org/openntf/domino/logging/LogHandlerOpenLog@0x00000000C0C365E0, entry count: 1)
4XESTACKTRACE at org/openntf/domino/logging/LogFilterHandler.publishConditionally(LogFilterHandler.java:352)
4XESTACKTRACE at org/openntf/domino/logging/LogFilterHandler.publishDefault(LogFilterHandler.java:342(Compiled Code))
4XESTACKTRACE at org/openntf/domino/logging/LogFilterHandler.publish(LogFilterHandler.java:312(Compiled Code))
5XESTACKTRACE (entered lock: org/openntf/domino/logging/LogFilterHandler@0x00000000C0C354A0, entry count: 1)
4XESTACKTRACE at java/util/logging/Logger.log(Logger.java:738(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.doLog(Logger.java:765(Compiled Code))
4XESTACKTRACE at java/util/logging/Logger.log(Logger.java:876(Compiled Code))
4XESTACKTRACE at org/openntf/domino/utils/DominoUtils$2.run(DominoUtils.java:450)
4XESTACKTRACE at java/security/AccessController.doPrivileged(AccessController.java:738(Compiled Code))
4XESTACKTRACE at org/openntf/domino/utils/DominoUtils.handleException(DominoUtils.java:444)
4XESTACKTRACE at org/openntf/domino/utils/DominoUtils.handleException(DominoUtils.java:411)
4XESTACKTRACE at org/openntf/domino/impl/DocumentCollection.getFirstDocument(DocumentCollection.java:136(Compiled Code))
4XESTACKTRACE at org/openntf/domino/iterators/DocumentCollectionIterator.(DocumentCollectionIterator.java:31(Compiled Code))
4XESTACKTRACE at org/openntf/domino/impl/DocumentCollection.iterator(DocumentCollection.java:763(Compiled Code))
4XESTACKTRACE at org/acme/kkom/dao/AttachmentDominoDAO.getAll(AttachmentDominoDAO.java:199(Compiled Code))
in bold is custom code, which itself is using this openntf framework
its always hard to further annotate: for example it may be the case that while the deadlock now occurs in this openntf framework, it may also be the case that to use the framework it is necessary to also make the functions/classes in the customer code synchronized.

[PatrickKwinten]

Release 12.0.1FP1 HF72 server release

[PatrickKwinten]

the java class that causes nullpointer exception

package org.acme.kkom.dao;

import java.io.File;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Enumeration;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Properties;
import java.util.TreeSet;
import java.util.Vector;
import java.util.logging.Level;

import javax.faces.context.FacesContext;

import org.apache.commons.io.FileUtils;
import org.openntf.domino.Database;
import org.openntf.domino.Document;
import org.openntf.domino.DocumentCollection;
import org.openntf.domino.EmbeddedObject;
import org.openntf.domino.Item;
import org.openntf.domino.Name;
import org.openntf.domino.RichTextItem;
import org.openntf.domino.Session;
import org.openntf.domino.View;
import org.openntf.domino.ViewEntry;
import org.openntf.domino.ViewEntryCollection;
import org.openntf.domino.utils.Factory;
import org.openntf.domino.utils.Factory.SessionType;
import org.openntf.domino.xsp.XspOpenLogUtil;

import com.ibm.commons.util.io.json.JsonException;
import com.ibm.commons.util.io.json.JsonJavaFactory;
import com.ibm.commons.util.io.json.JsonJavaObject;
import com.ibm.commons.util.io.json.JsonParser;
import com.ibm.xsp.extlib.util.ExtLibUtil;
import com.ibm.xsp.http.IUploadedFile;

import org.acme.kkom.app.ApplicationBean;
import org.acme.kkom.data.Attachment;
import org.acme.kkom.utils.JSFUtil;
import org.acme.kkom.utils.MultiGrowlMessages;
import org.acme.kkom.utils.Utils;

public class AttachmentDominoDAO implements AttachmentDAO, Serializable{

public static final long serialVersionUID = 1L;

private Utils utils;
private ApplicationBean appBean;

//Session with Authenticated User Access
private Session sess;
//Session with Full Admin Access
private Session sessFA;	

private Properties propDataSources;	
private Properties propStrings;	

private String filePath;
private String fileName;
	
public AttachmentDominoDAO(){
	utils = (Utils)ExtLibUtil.resolveVariable(FacesContext.getCurrentInstance(), "utilityBean");
	utils.printToConsole(this.getClass().getSimpleName().toString() + " // constructor");
	try {
		//ODA Enablement
		this.sess = Factory.getSession(SessionType.CURRENT);
		this.sessFA = Factory.getSession(SessionType.FULL_ACCESS);
		
		this.appBean = new ApplicationBean();			
		propDataSources = appBean.getPropDataSources();

		propStrings = utils.getPropertiesFile("matter.properties");
		
		filePath = utils.getFilePath();
		
		if(utils.validValueInPropertyFile(propDataSources,"db_inbox_filepath")){
			fileName = propDataSources.getProperty("db_inbox_filepath");
		}else{
			//fallback
			fileName = "inbox.nsf";
		}			
	} catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}
}	


/**
* Returns an Attachment object. 
* The Attachment object is a representation of the attached file in a NotesDocument.
* 
* @param	doc The NotesDocument to read data from in a view
* @return	The attached file for a Document. Max 1 attachment per document
* @see		 Attachment
* 
*/

private Attachment loadFromDocument(Document doc) {
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> unid = " + doc.getUniversalID());	
	Attachment attachment = new Attachment();
	try{		
		if (null != doc){
			attachment.setUnid(doc.getUniversalID());	
			attachment.setCreated(doc.getCreated().toJavaDate());				
			if(doc.hasItem("$FILE")){
				//Let's follow the instructions according to Domino Designer Help
				RichTextItem item = (RichTextItem)doc.getFirstItem("FILES");
				Vector<EmbeddedObject> v = item.getEmbeddedObjects();
				Enumeration<EmbeddedObject> e = v.elements();
				while (e.hasMoreElements()) {
					EmbeddedObject eo = (EmbeddedObject)e.nextElement();
					if(null != eo.getSource()) {
						if(null != eo.getSource()) {
							attachment.setFile(eo.getSource());
						} else {
							XspOpenLogUtil.logEvent(null, "Problem with reading attachment from entry " + doc.getUniversalID() + ", fileName.getName() returns " + eo.getSource(), Level.SEVERE, null);
						}

						if(null != eo.getSource() && !utils.Right(eo.getSource(),".").isEmpty()) {
							attachment.setExtension(utils.Right(eo.getSource(),"."));	
						} else {
							XspOpenLogUtil.logEvent(null, "Problem with reading attachment from entry " + doc.getUniversalID() + ", extension is empty for file " + eo.getSource(), Level.SEVERE, null);
						}

						attachment.setSizeHuman(FileUtils.byteCountToDisplaySize(eo.getFileSize()));							

						if(eo.getFileSize() > 0) {
							attachment.setSize(eo.getFileSize());
						} else {
							XspOpenLogUtil.logEvent(null, "Problem with reading attachment from entry " + doc.getUniversalID() + ", fileName.size() returns " + eo.getFileSize(), Level.SEVERE, null);
						}

						if(null != doc.getAuthors() && null != doc.getAuthors().firstElement()) {
							attachment.setCreator(doc.getAuthors().firstElement());
						} else {
							XspOpenLogUtil.logEvent(null, "Problem with reading attachment from entry " + doc.getUniversalID() + ", doc.getAuthors().firstElement() returns " + doc.getAuthors().firstElement(), Level.SEVERE, null);
						}

						String fieldName = "type";
						if (doc.hasItem(fieldName)) {
							attachment.setType(doc.getItemValueString(fieldName));
						}
					}else {
						XspOpenLogUtil.logEvent(null, "AttachmentDominoDAO - General problem during reading attachment from entry " + doc.getUniversalID() + " in database " + doc.getParentDatabase().getFileName() + ". eo.getSource() returns null.", Level.SEVERE, null);
					}			          
				}								
			}			
		}					
	} catch (Exception e) {			
		XspOpenLogUtil.logEvent(e, "General problem with reading attachment from entry in database" , Level.SEVERE, null);
	}
	return attachment;
}	


/**
* Returns an a list of Attachment objects. 
* The Attachment object is a representation of the attached file in a NotesDocument.
* 
* @param  key The string value to identify multiple documents in a view that match a column value. often the universal ID of the parent document
* @param  type Additional string value (often a category) to identify multiple documents in a view that match a column value
* @return      A list of Attachment objects
* @see         Attachment
* 
*/

public List<Attachment> getAll(String key, String type){
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> key = " + key + ", type = " + type);			
	ArrayList<Attachment> attachments = new ArrayList<Attachment>();		
	try {
		Database db = sess.getDatabase(null, filePath + fileName, false);			
		if (db.isOpen()){
			if(utils.validValueInPropertyFile(propDataSources,"vw_attach_unid_type")){
				View vw = db.getView(propDataSources.getProperty("vw_attach_unid_type"));
				if (null != vw){	
					//vw.refresh();
					//vw.setAutoUpdate(false);						
					if (null != key || null != type){
						DocumentCollection coll = vw.getAllDocumentsByKey(key + "#" + type, true);
						if(coll.size() > 0){
							for (Document doc : coll){
								if(!doc.isDeleted()){
									Attachment attachment = loadFromDocument(doc);
									attachments.add(attachment);
								}
							}
						}
					}						
					//vw.setAutoUpdate(true);						
				}
			}			
		}	
	} catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}		
	
	if(attachments.size() > 0){
		utils.printToConsole("Number of attachments found for " + key + ": " + attachments.size());
	}
	
	ExtLibUtil.getViewScope().put("attachments" + key + type, attachments);
	return attachments;
}


/**
* Returns a string reference (absolute url) for an attached file in a NotesDocument. 
* The Attachment object is a representation of the attached file in a NotesDocument.
* 
* @param  unid The universal ID of a NotesDocument
* @return      A string reference (url) for an attached file in a NotesDocument.
* 
*/
@Override
public String getAttachmentUrl(String unid) {
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> unid = " + unid);
	String url = null;
	try{
		if(null != unid){
			Database db = sess.getDatabase(null, filePath + fileName, false);
			if (db.isOpen()){
				Document doc = db.getDocumentByUNID(unid);
				if(null != doc){
					if(null != doc.getUniversalID()){
						Vector<Object> att = sess.evaluate("@AttachmentNames", db.getDocumentByUNID(unid));
						if (att.get(0) != ""){
							String protocol = JSFUtil.getXSPContext().getUrl().getScheme();
							String host = JSFUtil.getXSPContext().getUrl().getHost();
							url = protocol + "://" + host + "/" + db.getFilePath().replace("\\","/") + "/0/" + unid + "/$FILE/" + att.get(0);
						}
					}
				}
			}
		}
	}catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}
	utils.printToConsole("url = " + url);
	return url;
}

/**
* Returns a string reference (absolute url) for an attached file in a NotesDocument. 
* The Attachment object is a representation of the attached file in a NotesDocument.
* 
* @param  unid The universal ID of a NotesDocument
* @return      A string reference (url) for an attached file in a NotesDocument.
* 
*/
@Override
public String getAttachmentUrlMinuteFile(String unid) {
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> unid = " + unid);
	String url = null;
	try{
		if(null != unid){
			Map<String, Object> sesScope = JSFUtil.getSessionScope();
			if(sesScope.containsKey("activeCommittee")) {
				String activeCommittee = (String) sesScope.get("activeCommittee");			
				Database db = sess.getDatabase(null, filePath + activeCommittee, false);
				if (db.isOpen()){
					Document doc = db.getDocumentByUNID(unid);
					if(null != doc){
						if(null != doc.getUniversalID()){
							Vector<Object> att = sess.evaluate("@AttachmentNames", db.getDocumentByUNID(unid));
							if (att.get(0) != ""){
								String protocol = JSFUtil.getXSPContext().getUrl().getScheme();
								String host = JSFUtil.getXSPContext().getUrl().getHost();
								url = protocol + "://" + host + "/" + db.getFilePath().replace("\\","/") + "/0/" + unid + "/$FILE/" + att.get(0);
							}
						}
					}else {
						XspOpenLogUtil.logEvent(null, "Doc is null", Level.SEVERE, null);
					}
				}else {
					XspOpenLogUtil.logEvent(null, "Cannot open database under " + filePath + activeCommittee, Level.SEVERE, null);
				}
			}else {
				XspOpenLogUtil.logEvent(null, "Sessionscope does not contain key activeCommittee", Level.SEVERE, null);
			}
			
		}
	}catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}
	utils.printToConsole("url = " + url);
	return url;
}


/**
* Returns the number of Attachment objects for given keys. 
* The Attachment object is a representation of the attached file in a NotesDocument.
* 
* @param  key The universal ID of a NotesDocument. Mostly the UNID of the parent document.
* @param  type The category a NotesDocument is stored under.
* @return      A number (integer)
* 
*/
@Override
public int getNumberOfAttachments(String key, String type){
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> key = " + key + ", type = " + type);
	int total = 0;
	try {			
		Database db = sess.getDatabase(null, filePath + fileName, false);
		if (db.isOpen()){
			if(utils.validValueInPropertyFile(propDataSources,"vw_attach_unid_type")){
				View vw = db.getView(propDataSources.getProperty("vw_attach_unid_type"));
				if (null != vw){
					//vw.refresh();
					//vw.setAutoUpdate(false);						
					if (null != key){							
						if(null != type){
							if(!key.isEmpty() && !type.isEmpty()){
								DocumentCollection coll = vw.getAllDocumentsByKey(key + "#" + type, true);	
								if(coll.size() > 0){
									total = coll.size();	
								}	
								
								/*ViewEntryCollection entries = vw.getAllEntriesByKey(key + "#" + type, true);	
								if (null != entries) {						
									total = entries.getCount();						
								}*/
							}
						}
					}						
					//vw.setAutoUpdate(true);					
				}			
			}		
		}	
	} catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}
	utils.printToConsole("Number of attachments found for " + key + ": " + total);
	return total;
}

@Override
public void remove(String unid){
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> unid = " + unid);
	try {
		Database db = sess.getDatabase(null, filePath + fileName, false);
		if (db.isOpen()){
			Document doc = db.getDocumentByUNID(unid);
			if (null != doc) { 
				try{
					if (doc.remove(true)){
						utils.printToConsole("document removed -> unid = " + unid);
						if(utils.validValueInPropertyFile(propDataSources,"vw_attach_unid_type")){
							View vw = db.getView(propDataSources.getProperty("vw_attach_unid_type"));
							if (null != vw){	
								vw.refresh();	
							}							}
					}else{
						utils.printToConsole("document could not be removed -> unid = " + unid);
					}
				} catch(Exception e){
					XspOpenLogUtil.logError(e);
				}
			}				
		}		
	} catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}	
}


/**
* Updates for the field "Officers" the content and type in a NotesDocument
* 
* @param  key The universal ID of a NotesDocument to allocate it. Mostly the UNID of the parent document.
* @param  fieldType The type of field. Readers or Authors.
* @param  names The content for the field. A list of names.
* 
*/	
@Override
public void updateOfficers(String key, String fieldType, Vector<String> names){
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> key = " + key + ", fieldType = " + fieldType);
	try {			
		Database db = sess.getDatabase(null, filePath + fileName, false);
		if (db.isOpen()){		
			if(utils.validValueInPropertyFile(propDataSources,"vw_attach_unid_type")){
				View vw = db.getView(propDataSources.getProperty("vw_attach_unid_type"));
				if (null != vw){	
					//vw.refresh();	
					//vw.setAutoUpdate(false);
					
					DocumentCollection coll = db.createDocumentCollection();
					if (null != key){
						coll = vw.getAllDocumentsByKey(key, true);						
					}									
					
					if(coll.size() > 0){
						//ODA way:
						for (Document doc : coll) {
							if(doc.isValid()){
								String fieldName = "Officers";							
								if (names.size() > 0){
									Item field = doc.replaceItemValue(fieldName, names);
									if(fieldType.equals("Authors")){
										field.setAuthors(true);
									} else{
										field.setReaders(true);
									}
								} else{
									doc.removeItem(fieldName);
								}							
								if(doc.save(true,false)){
									utils.printToConsole("document saved");
									vw.refresh();	
								}else{
									utils.printToConsole("document could not be saved");
								}
							}												
						}
					}
										
					
					
					/*ViewEntryCollection entries = vw.createViewEntryCollection();
					if (null != key){
						entries = vw.getAllEntriesByKey(key, true);						
					}
					else{
						entries = vw.getAllEntries();
					}				
					//ODA way:
					for (ViewEntry entry : entries) {
						if(entry.isValid()){
							if(entry.isDocument()){
								Document doc = entry.getDocument();
								String fieldName = "Officers";							
								if (names.size() > 0){
									Item field = doc.replaceItemValue(fieldName, names);
									if(fieldType.equals("Authors")){
										field.setAuthors(true);
									} else{
										field.setReaders(true);
									}
								} else{
									doc.removeItem(fieldName);
								}							
								if(doc.save(true,false)){
									vw.refresh();	
								}
							}								
						}							
					}*/
					//vw.setAutoUpdate(true);					
				}
			}
		}	
	} catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}		
}


/**
* Updates for the field "Reciters" the content and type in a NotesDocument
* 
* @param  key The universal ID of a NotesDocument to allocate it. Mostly the UNID of the parent document.
* @param  fieldType The type of field. Readers or Authors.
* @param  names The content for the field. A list of names.
* 
*/	
@Override
public void updateReciters(String key, String fieldType, Vector<String> names){
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> key = " + key + ", fieldType = " + fieldType);		
	try {			
		Database db = sess.getDatabase(null, filePath + fileName, false);
		if (db.isOpen()){	
			if(utils.validValueInPropertyFile(propDataSources,"vw_attach_unid_type")){
				View vw = db.getView(propDataSources.getProperty("vw_attach_unid_type"));
				if (null != vw){	
					//vw.refresh();	
					//vw.setAutoUpdate(false);
					
					DocumentCollection coll = db.createDocumentCollection();
					if (null != key){
						coll = vw.getAllDocumentsByKey(key, true);						
					}
					
					if(coll.size() > 0){
						//ODA way:
						for (Document doc : coll) {
							if(doc.isValid()){
								String fieldName = "Reciters";							
								if (names.size() > 0){
									Item field = doc.replaceItemValue(fieldName, names);
									if(fieldType.equals("Authors")){
										field.setAuthors(true);
									} else{
										field.setReaders(true);
									}
								} else{
									doc.removeItem(fieldName);
								}							
								
								if(doc.save(true,false)){
									utils.printToConsole("document saved");
									vw.refresh();	
								}else{
									utils.printToConsole("document could not be saved");
								}
							}							
						}	
					}
					
					ViewEntryCollection entries = vw.createViewEntryCollection();
					if (null != key){
						entries = vw.getAllEntriesByKey(key, true);						
					}
					else{
						entries = vw.getAllEntries();
					}
					//ODA way:
					for (ViewEntry entry : entries) {
						if(entry.isValid()){
							if(entry.isDocument()){
								Document doc = entry.getDocument();
								String fieldName = "Reciters";							
								if (names.size() > 0){
									Item field = doc.replaceItemValue(fieldName, names);
									if(fieldType.equals("Authors")){
										field.setAuthors(true);
									} else{
										field.setReaders(true);
									}
								} else{
									doc.removeItem(fieldName);
								}							
								
								if(doc.save(true,false)){
									utils.printToConsole("document saved");
									vw.refresh();	
								} else{
									utils.printToConsole("document could not be saved");
								}
							}
						}							
					}						
					//vw.setAutoUpdate(true);				
				}
			}				
		}	
	} catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}		
}

/**
* Updates for the field the content and type in a NotesDocument
* 
* @param  key The universal ID of a NotesDocument to allocate it. Mostly the UNID of the parent document.
* @param  item The name of the NotesItem to copy
* @param  fieldName Name of the NotesField 
* 
*/	
@Override
public void updateAccess(String key, Item item, String fieldName) {
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> key = " + key + ", fieldName = " + fieldName);		
	try {			
		Database db = sessFA.getDatabase(null, filePath + fileName, false);
		if (db.isOpen()){		
			if(utils.validValueInPropertyFile(propDataSources,"vw_attach_unid")){
				View vw = db.getView(propDataSources.getProperty("vw_attach_unid"));
				if (null != vw){	
					//vw.refresh();	
					//vw.setAutoUpdate(false);
					
					DocumentCollection coll = db.createDocumentCollection();
					if (null != key){
						coll = vw.getAllDocumentsByKey(key, true);					
					}
					
					//ODA way:
					if(coll.size() > 0){
						for (Document doc : coll) {
							if(doc.isValid()){
								if(null != item){
									doc.copyItem(item);
								} else{
									doc.removeItem(fieldName);
								}							
								if(doc.save(true,false)){
									utils.printToConsole("document saved");
									vw.refresh();	
								} else{
									utils.printToConsole("document¨could not be saved");
								}
							}							
						}
					}						
					
					/*ViewEntryCollection entries = vw.createViewEntryCollection();
					if (null != key){
						entries = vw.getAllEntriesByKey(key, true);					
					}
					else{
						entries = vw.getAllEntries();
					}
					//ODA way:
					for (ViewEntry entry : entries) {
						if(entry.isValid()){
							if(entry.isDocument()){
								Document doc = entry.getDocument();								
								if(null != item){
									doc.copyItem(item);
								} else{
									doc.removeItem(fieldName);
								}							
								if(doc.save(true,false)){
									vw.refresh();	
								}
							}
						}							
					}*/
					//vw.setAutoUpdate(true);					
				}
			}
		}	
	} catch (Exception e) {
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	}	
}

@Override
public void save(String parentId, String type, String fields, IUploadedFile iUploadedFile) {
	String methodName = new Object(){}.getClass().getEnclosingMethod().getName();
	utils.printToConsole(this.getClass().getSimpleName().toString() + " " + methodName + " -> parentId = " + parentId + ", type = " + type);		
	try {
		if (null != iUploadedFile) {
			Database db = sess.getDatabase(null, filePath + fileName, false);
			if (db.isOpen()){
				String fieldName = null;
				//get the uploaded file
				//IUploadedFile iUploadedFile = uploadedFile.getUploadedFile();

				//get the original filename
				String tempClientFile = iUploadedFile.getClientFileName();
				utils.printToConsole("# iUploadedFile - getClientFileName: " + iUploadedFile.getClientFileName());
				utils.printToConsole("# iUploadedFile - getServerFileName:  " + iUploadedFile.getServerFileName());
				utils.printToConsole("# iUploadedFile - getContentType:  " + iUploadedFile.getContentType());
				utils.printToConsole("# iUploadedFile - getContentLength:  " + iUploadedFile.getContentLength());

				//get the server file (with a cryptic filename)
				File tempFile = iUploadedFile.getServerFile();					
				utils.printToConsole("# tempFile exist: " + tempFile.exists());	
				if(!tempFile.exists()){
					XspOpenLogUtil.logErrorEx(null, "tempFile " + tempFile + " does not exist for user " + sess.getUserName(), Level.SEVERE, null);
				}
				utils.printToConsole("# tempFile canread: " + tempFile.canRead());
				if(!tempFile.canRead()){
					XspOpenLogUtil.logErrorEx(null, "tempFile " + tempFile + " not readable for user " + sess.getUserName(), Level.SEVERE, null);
				}
				utils.printToConsole("# tempFile canwrite: " + tempFile.canWrite());
				if(!tempFile.canWrite()){
					XspOpenLogUtil.logErrorEx(null, "tempFile " + tempFile + " not writable for user " + sess.getUserName(), Level.SEVERE, null);
				}
				utils.printToConsole("# tempFile ishidden: " + tempFile.isHidden());
				if(tempFile.isHidden()){
					XspOpenLogUtil.logErrorEx(null, "tempFile " + tempFile + " is hidden for user " + sess.getUserName(), Level.SEVERE, null);
				}
				
				File correctedFile = new File( tempFile.getParentFile().getAbsolutePath() + File.separator + tempClientFile );
				utils.printToConsole("# correctedFile getName: " + correctedFile.getName());
				utils.printToConsole("# correctedFile getPath: " + correctedFile.getPath());
				utils.printToConsole("# correctedFile getAbsolutePath: " + correctedFile.getAbsolutePath());
				utils.printToConsole("# correctedFile getCanonicalPath: " + correctedFile.getCanonicalPath());

				boolean success = false;					
				try{
                    success = tempFile.renameTo(correctedFile);
				} catch (Exception e){
				    XspOpenLogUtil.logErrorEx(e, "Could not rename temp file " + tempFile + " to " + correctedFile + " for user " + sess.getUserName(), Level.SEVERE, null);
				}					
				
				if (success) {
					//do whatever you want here with correctedFile
					String borreSummary = null;
					Document parentDoc = db.getDocumentByUNID(parentId);
					if (null != parentDoc){
						if(type.trim().equalsIgnoreCase(parentId.trim())) {
							String newParentID = parentDoc.getItemValueString("parentId");								
							parentDoc = db.getDocumentByUNID(newParentID);
						}
						
						fieldName = "fldKundSammandrag";
						if (parentDoc.hasItem(fieldName)){
							borreSummary = parentDoc.getItemValueString(fieldName);
						}
					}					

					Document doc = db.createDocument();
					doc.appendItemValue("Form","fa_Attachment");
					doc.appendItemValue("parentId", parentId);
					doc.appendItemValue("type", type);

					Name origAuthor = sess.createName(sess.getEffectiveUserName());
											
					//get the groups the user belongs to
					/*Vector<Name> groupNameList = sess.getUserGroupNameList();
					Vector<String> groups = new Vector<String>();
					for (int i = 0; i < groupNameList.size(); i++) {
						Name groupName = groupNameList.elementAt(i);
						groups.add(groupName.getAbbreviated()); 
					}*/
					Collection<String> groupNameColl = sess.getUserGroupNameCollection();
					Vector<String> groups = new Vector<String>();
					Iterator<String> iterator = groupNameColl.iterator();
					while (iterator.hasNext()){
						Name groupName = sess.createName(iterator.next());							
						groups.add(groupName.getCommon());
					}						

					Vector<String> vecAuthors = new Vector<String>(4);
					vecAuthors.add("[Admin]");
					vecAuthors.add("[SuperAdmin]");
					vecAuthors.add("[SuperDuper]");
					if(groups.contains("KKOM_OpAdmin")){
						vecAuthors.add("KKOM_OpAdmin");	
					}
					if(groups.contains("KKOM_KYC Center")){
						vecAuthors.add("KKOM_KYC Center");	
					}
					if(groups.contains("KKOM_MidCorp_Riga")){
						vecAuthors.add("KKOM_MidCorp_Riga");	
					}
					if(groups.contains("KKOM_MidCorp_Sthlm")){
						vecAuthors.add("KKOM_MidCorp_Sthlm");	
					}
					if (null!=parentDoc) {
						if (parentDoc.hasItem("fldKontorChefFName")) {
							if(!parentDoc.getItemValueString("fldKontorChefFName").trim().equalsIgnoreCase("")){
								vecAuthors.add(parentDoc.getItemValueString("fldKontorChefFName").trim());
							}
						}
						if (parentDoc.hasItem("fldKundsvarig")) {
							if(!parentDoc.getItemValueString("fldKundsvarig").trim().equalsIgnoreCase("")){
								vecAuthors.add(parentDoc.getItemValueString("fldKundsvarig").trim());
							}
						}									
					}
					vecAuthors.add(origAuthor.getCanonical());
					
					String access;
					access = utils.getAccessGroup(borreSummary);
					vecAuthors.add("KKOM_" + access + "_Admin");

					TreeSet<String> uniqueAuthors = new TreeSet<String>(vecAuthors);
					vecAuthors = new Vector<String>(uniqueAuthors);

					Item authorRole = doc.replaceItemValue("Authors", vecAuthors);
                    authorRole.setAuthors(true);
                   
                    fieldName = "Officers";
					if (null != parentDoc) {
						if (parentDoc.hasItem(fieldName)){
							Item item = parentDoc.getFirstItem(fieldName);
							doc.copyItem(item, null);
						}
					}
					
					fieldName = "Reciters";
					if (null != parentDoc) {
						if (parentDoc.hasItem(fieldName)){
							Item item = parentDoc.getFirstItem(fieldName);
							doc.copyItem(item, null);
						}
					}
					
					fieldName = "Readers";
					if (null!=parentDoc) {
						if (parentDoc.hasItem(fieldName)){
							Item item = parentDoc.getFirstItem(fieldName);
							doc.copyItem(item, null);
						}
					}
					
					fieldName = "Support";
					if(null != borreSummary){
						if (parentDoc.hasItem(fieldName)){
							Item item = parentDoc.getFirstItem(fieldName);
							doc.copyItem(item, null);
						}		
					}

					fieldName = "files";
					RichTextItem rtFiles = doc.createRichTextItem(fieldName);				  
					rtFiles.embedObject(lotus.domino.EmbeddedObject.EMBED_ATTACHMENT, "", correctedFile.getAbsolutePath(), null);
					try {
						JsonJavaObject objJson = (JsonJavaObject) JsonParser.fromJson(JsonJavaFactory.instanceEx, fields);
						if (null != objJson){
							if(objJson.containsKey("fields")){
								ArrayList<JsonJavaObject> listFields = (ArrayList<JsonJavaObject>) objJson.get("fields");		
								if(null != listFields){
									
									for(int i=0; i<listFields.size(); i++){
										JsonJavaObject fieldItem = listFields.get(i);
										if(fieldItem.containsKey("Name")){
											String itemRef = (String) fieldItem.get("name");
											if (parentDoc.hasItem(itemRef)){
												Item item = parentDoc.getFirstItem(itemRef);
												doc.copyItem(item, null);
												Item docItem = doc.getFirstItem(itemRef);
												if(fieldItem.containsKey("type")){
													String fieldType = (String) fieldItem.get("type");
													if (fieldType.equals("Names")){
														docItem.setNames(true);
													}
													if (fieldType.equals("Authors")){
														docItem.setAuthors(true);
													}
													if (fieldType.equals("Readers")){
														docItem.setReaders(true);
													}
													if (fieldType.equals("Text")){
														docItem.setSummary(true);
													}
												}													
											}
										}											
									}
								}
							}								
						}
					} catch (JsonException e1) {
						XspOpenLogUtil.logErrorEx(e1, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
					}
					
					if(doc.save()){
						utils.printToConsole("document saved");
						//new!
						if(utils.validValueInPropertyFile(propDataSources,"vw_attach_unid_type")){
							View vw = db.getView(propDataSources.getProperty("vw_attach_unid_type"));
							if (null != vw){	
								vw.refresh();	
							}
						}
					}else{
						utils.printToConsole("document could not be saved");
					}
					
					//set a viewscope with doc UNID of latest uploaded file for whatever purpose
					ExtLibUtil.getViewScope().put("attachmentUnid", doc.getUniversalID());						

					//if we're done: rename it back to the original filename, so it gets cleaned up by the server
					utils.printToConsole("# correctedFile exists: " + correctedFile.exists());
					if(!correctedFile.exists()){
						XspOpenLogUtil.logErrorEx(null, "correctedFile " + correctedFile + " does not exist for user " + sess.getUserName(), Level.SEVERE, null);
					}
					utils.printToConsole("# correctedFile canread: " + correctedFile.canRead());
					if(!correctedFile.canRead()){
						XspOpenLogUtil.logErrorEx(null, "correctedFile " + correctedFile + " is not readable for user " + sess.getUserName(), Level.SEVERE, null);
					}
					utils.printToConsole("# correctedFile canwrite: " + correctedFile.canWrite());
					if(!correctedFile.canWrite()){
						XspOpenLogUtil.logErrorEx(null, "correctedFile " + correctedFile + " is not writablee for user " + sess.getUserName(), Level.SEVERE, null);
					}
					if(correctedFile.isHidden()){
						XspOpenLogUtil.logErrorEx(null, "correctedFile " + correctedFile + " is hidden for user " + sess.getUserName(), Level.SEVERE, null);
					}
					utils.printToConsole("# correctedFile ishidden: " + correctedFile.isHidden());
					try{
                       correctedFile.renameTo(tempFile);
					} catch (Exception e){
						String msg = null;
						if(utils.validValueInPropertyFile(propStrings,"attachment_growl_file_upload_error")){
							msg = propStrings.getProperty("attachment_growl_file_upload_error");
						}
						MultiGrowlMessages.createGrowlMessage(utils.formatPropertyValueToUTF(msg), "warning");							
					    XspOpenLogUtil.logErrorEx(e, "Could not rename correctedFile file " + correctedFile + " to " + correctedFile + " for user " + sess.getUserName(), Level.SEVERE, null);
					}

					//rtFiles.recycle();
					
					String msg = null;
					if(utils.validValueInPropertyFile(propStrings,"attachment_growl_file")){
						msg = propStrings.getProperty("attachment_growl_file");
					}						
					MultiGrowlMessages.createGrowlMessage(utils.formatPropertyValueToUTF(msg) + iUploadedFile.getClientFileName(), "success");
					
					if(null != tempFile && tempFile.canWrite()){
						tempFile.delete();
					}
					if(null != correctedFile && correctedFile.canWrite()){
						correctedFile.delete();
					}
				}
				else{
					String msg = null;
					if(utils.validValueInPropertyFile(propStrings,"attachment_growl_file_upload_error")){
						msg = propStrings.getProperty("attachment_growl_file_upload_error");
					}						
					MultiGrowlMessages.createGrowlMessage(msg, "warning");						
					XspOpenLogUtil.logErrorEx(null, "Could not rename file " + correctedFile + " to " + tempFile + " for user " + sess.getUserName(), Level.SEVERE, null);
				}
			}		 
		}else{
			String msg = null;
			if(utils.validValueInPropertyFile(propStrings,"attachment_growl_no_file")){
				msg = propStrings.getProperty("attachment_growl_no_file");
			}				
			MultiGrowlMessages.createGrowlMessage(utils.formatPropertyValueToUTF(msg), "error");
		}
	} catch (Exception e) {
		String msg = null;
		if(utils.validValueInPropertyFile(propStrings,"attachment_growl_file_upload_error")){
			msg = propStrings.getProperty("attachment_growl_file_upload_error");
		}			
		MultiGrowlMessages.createGrowlMessage(utils.formatPropertyValueToUTF(msg), "warning");	
		XspOpenLogUtil.logErrorEx(e, JSFUtil.getXSPContext().getUrl().toString(), Level.SEVERE, null);
	} 
}

}

[jesse-gallagher]

Are you able to reproduce the trouble in a minimal case? To try to see it here, I:

• Created a new, empty DB
• Checked just ODA and the ExtLib as XPages library dependencies
• Created a view named "Docs" with "FirstName" and "LastName" columns
• Populated that with about a dozen docs
• Made an XPage that just does <xp:text value="${javascript:new bean.BeanGuy().doFoo()}"/>
• Made this Java class that does an incomplete variant of your original example:

package bean;

import org.openntf.domino.Database;
import org.openntf.domino.Session;
import org.openntf.domino.View;
import org.openntf.domino.ViewEntryCollection;
import org.openntf.domino.ViewEntry;

import org.openntf.domino.utils.Factory;
import org.openntf.domino.utils.Factory.SessionType;

import com.ibm.xsp.extlib.util.ExtLibUtil;

public class BeanGuy {
	public String doFoo() {
		
		Session session = Factory.getSession(SessionType.CURRENT);
		Database database = Factory.getWrapperFactory().fromLotus(ExtLibUtil.getCurrentDatabase(), Database.SCHEMA, session);
		String key = "";
		
		View vw = database.getView("Docs");
		if (null != vw){
		    vw.setAutoUpdate(false);
		    ViewEntryCollection entries = vw.createViewEntryCollection();
		    if (null != key) {
		        if(!key.isEmpty()){
		            entries = vw.getAllEntriesByKey(key,true);
		        } else {
		            entries = vw.getAllEntries();
		        }
		    } else {
		        entries = vw.getAllEntries();
		    }
		    for (ViewEntry entry : entries) {
		        if(!entry.isCategory()){
		            System.out.println(entry.getColumnValues());
		        }
		    }
		    vw.setAutoUpdate(true);
		}
		return session.toString();
	}
}

In my case, this worked fine (regardless of whether or not I filled key in with a key), though I'm on Domino and ODA 12.0.2. Could you try that on your server to see if that alone trips it up? That'll be a good baseline for trying to find the problem.

I'll attach my minimal NSF here:
oda186.nsf.zip

[PatrickKwinten]

My apologies, I was out of office for home renovation.

I have tested the NSF and I have no problems running the foo xsp

[perlausten]

Do you also get the NPE error if you use vw.setAutoUpdate(false); before and vw.setAutoUpdate(true); after the for loop? I see in your pasted code that setAutoUpdate is not used.