RAM drive not available again in lucee(RAM error)

Dick_Goosen · December 13, 2016, 5:34am

Hi,

By the way this morning again the error. If you are quick see it on www.franimo.nl/test2/

Regards,

Dick Goosen

dnando · December 13, 2016, 11:29am

I wonder if a shared hosting provider is managing this server …

Is this the case? If so, it might be that this app is starved for memory no
matter how the JVM opts are set.

Aria Media Sagl
+41 91 600 9601
Skype: ariamedia

Dick_Goosen · December 13, 2016, 12:01pm

Forgot to use it. Just to be quick.

Dick_Goosen · December 13, 2016, 12:14pm

There is another function which uses the RAM, it is practically the same thing. Code:

<cffunction name="createImage" returnType="string" access="public" hint="uploads, resizes and puts an image in the database">
		<cfargument name="propertyId" type="numeric" required="false" default="0">
		<cfargument name="agentId" type="numeric" required="false" default="5">
		<cfargument name="uploadedFile" type="any" required="true">
		<cfargument name="newImageWidth" type="numeric" required="true">
		<cfargument name="imageType" type="string" default="" required="false">
		<cfargument name="imageName" type="string" default="" required="false">
		
		<cffile action="upload"
		filefield="#arguments.uploadedFile#" 
		destination="ram://"
		accept = "image/jpg,image/jpeg,image/pjpeg,image/gif,image/png"
		nameconflict="makeunique" />
		
		<cfif imageName is "">
			<!--- create unique image name if no image name is provided --->
			<cfset imgName = lCase(listFirst(CreateUUID(), "-")) & right(TimeFormat(Now(),"hhmmss"), 3)>
			<cfset imgName=ListAppend(imgName,file.ServerFileExt,".")>
			<cfelse>
			<cfset imgName=arguments.imageName>
		</cfif>
		
		<cfset uploadedFile="#file.ServerFile#">
		
		<!---Now rename the image and store it in the appropriate folder
		<cffile action="rename"
				source="#request.wwwroot_folder#/images/pictures/#year(now())#/#month(now())#/#file.ServerFile#"
				destination="#request.wwwroot_folder#/images/pictures/#year(now())#/#month(now())#/#imgName#"> --->
				
		<!--- Read the Uploaded file to memory and give it a variable name of  'mainImage' 
		<cfimage name="mainImage" source="#request.wwwroot_folder#/images/pictures/#year(now())#/#month(now())#/#arguments.uploadedFile#">--->

		<!--- Create thumbnail, note this does not resize the image in memory, just the image that is  written to disk.--->
		<cfset cfImage = imageNew("ram://#file.ServerFile#")>
		<cfset imageData = imageInfo(cfImage)>

		<cfset imageWidth=imageData.width>
		<cfset imageHeight=imageData.height>
		
		<cfset theDivisionNewImage=arguments.newImageWidth/imageWidth>
		<cfset newImageHeight=round(imageHeight*theDivisionNewImage)>
		
		
		<cfparam name="imageTypeId" default="1">
		
		<cfswitch expression="#arguments.imageType#">
			<cfcase value="thumb">
				<!--- Check if main thumb is already there --->
				<cfquery datasource="#request.datasourceName#" name="checkMainThumb">
				SELECT * FROM pictures p WHERE picture_type_id=4 and property_id =#arguments.propertyId#
				</cfquery>
				<cfif checkMainThumb.recordCount IS 0>
					<cfset imageTypeId="4">
				<cfelse>
					<cfset imageTypeId="2">
				</cfif>
			</cfcase>
			<cfcase value="logo">
				<cfset imageTypeId="3">
			</cfcase>
		</cfswitch>
		
		<cfset newImageName= "#imageType##imgName#">
		
		<!--- create main image --->
		<cfimage action="resize"
		width="#newImageWidth#"
		height="#newImageHeight#"
		source="ram://#file.ServerFile#"
		destination="#request.wwwroot_folder#/images/pictures/#year(now())#/#month(now())#/#newImageName#"
		overwrite="true">
		
		<!--- <cffile action = "delete" file = "#request.wwwroot_folder#/images/pictures/#year(now())#/#month(now())#/#uploadedFile#">--->
		
		<cffile action="delete" file="ram://#file.ServerFile#">
		
		<cfquery datasource="#request.datasourceName#" name="getMaxSortOrder">
			SELECT MAX(sort_order) AS sortOrder FROM pictures WHERE property_id = <cfqueryparam cfsqltype="cf_sql_integer" value="#arguments.propertyId#">
		</cfquery>
		
		<cfif getMaxSortOrder.sortOrder is not "">
			<cfset nextSortOrder=getMaxSortOrder.sortOrder +1>
		<cfelse>
			<cfset nextSortOrder=1>
		</cfif>
		
		<cfquery datasource="#request.datasourceName#" name="insertProperty">
		INSERT INTO pictures (name,width,height,property_id,picture_type_id,agent_id,sort_order)
		VALUES (<cfqueryparam cfsqltype="cf_sql_varchar" value="#year(now())#/#month(now())#/#newImageName#">,<cfqueryparam cfsqltype="cf_sql_integer" value="#newImageWidth#">,<cfqueryparam cfsqltype="cf_sql_integer" value="#newImageHeight#">,<cfqueryparam cfsqltype="cf_sql_integer" value="#arguments.propertyId#">, <cfqueryparam cfsqltype="cf_sql_integer" value="#imageTypeId#">,<cfqueryparam cfsqltype="cf_sql_integer" value="#arguments.agentId#">,<cfqueryparam cfsqltype="cf_sql_integer" value="#nextSortOrder#">)
		</cfquery>
		
		<cfreturn imgName>
		
	</cffunction>

These are all the functions I use regarding the RAM drive.

Regards,

Dick Goosen

Dick_Goosen · December 13, 2016, 5:25am

Hi,

Thank you for your quick response. I will give this to the hosting guys who take care of the server.

Regards,

Dick Goosen

Dick_Goosen · December 21, 2016, 6:37am

Is there any update on this. I still got the error. See Uw statistieken http://www.franimo.nl/test2/ How can the RAM suddenly disappear?

Regards,

Dick Goosen

micstriit · August 23, 2019, 9:59am

Is there a jira ticket for it?

micstriit · August 28, 2019, 4:54pm

i have found the reason for this issue. in case Lucee runs out of memory it starts to delete files from the ram cache, actually the JVM dies it, because we are using soft references.
Problem is, this does not happen in a organized matter.
the JVM should start deleteting file on top of the tree, so no elements (directories) that have children.
in this case the root directory “/” is deleted and the Ram Resource cannot recover from this.

workaround, make sure not to use to many files in the ram cache.

micstriit · August 28, 2019, 4:59pm

see [LDEV-1109] - Lucee for more details

bdw429s · August 28, 2019, 5:11pm

I never knew the RAM drive was a soft reference store. I would have expected any files written there would always remain like a normal hard drive. I’m not sure I like that and I bet most people using it don’t account for that like you would when using a cache. I’d rather control what gets deleted from ram drive and when so I can choose the files my app still needs. I don’t trust Lucee or the JVM to properly “guess” what files I’m finished working with.

pfreitag · August 28, 2019, 8:36pm

I agree with Brad here, and I also did not know it was a soft reference (as I’m sure most would not expect).

I’d rather it throw an OutOfMemoryError or something than delete files I might need.

bdw429s · August 28, 2019, 8:45pm

@Abram_Adams @abramadams is there a chance this explanation above is the reason Trycf.com has been randomly failing for a while. it’s been random paths on ram disk going away.

pfreitag · August 28, 2019, 8:45pm

So given this code:

if ( fileExists("ram://file.txt") ) {
   //low on memory, ram drive del *.*
   x = fileRead("ram://file.txt");
}

It would then be possible to have a FileNotFound exception?!?

This makes the ram drive pretty unusable / unstable. I’d rather crash the server if I forget to delete my files from ram drive (because at least I can fix that in my code) instead of having the server delete ram files willy nilly.

bdw429s · August 28, 2019, 8:49pm

Yeah, that’s what I was thinking Pete. The current behavior seems dangerous. I appreciate the attempt to keep memory, but if I have an issue where ramdisk is filling up memory, I’d rather get the OOM error that lets me know to analyze my code and fix it. This may keep my server online, but with quirky hard to track bugs with files that disappear. @micstriit Can this be configured so ram disk is persistent?

Zackster · August 28, 2019, 9:12pm

There’s a similar issue with cache clearing too

bdw429s · August 28, 2019, 9:16pm

Cache eviction policies (and timeouts) are more of an expected behavior though. That’s why most cache access will be in the form of “get if exists, otherwise create” so I’m more ok with a cache item disappearing than a file on a file system.

Zackster · August 28, 2019, 9:59pm

Except Lucee doesn’t recover from evicted cache items, try the externalized strings option with a large enough code base to exceed the hardwired 1000 item limit and basic stuff like cfdump breaks.

bdw429s · August 28, 2019, 10:35pm

That sounds like a bug in Lucee’s usage of the cache then. Possibly from back when the RamCache used to not have any eviction policy. Is there a ticket for that?

Zackster · August 28, 2019, 10:42pm

There’s both tickets about the externalized strings issue and allowing the cache size to be configurable, not sure if there’s a ticket about recovering from missing cache items. There’s also a random blank page bug since 5.3.2 which is probably related.

crash on missing externalize txt file under cfclasses
https://luceeserver.atlassian.net/browse/LDEV-2354

allow configuring maximum cache, temp and template cache sizes
https://luceeserver.atlassian.net/browse/LDEV-2418

Page not rendered until file saved without changes
https://luceeserver.atlassian.net/browse/LDEV-2376

Gary_Stanton · June 2, 2020, 3:13pm

Looks like I’m falling foul of this issue too.
So, the way I read this, the RAM disk is unusable… the TryCF example fails intermittently in both Lucee and ACF, so I guess it’s not a Lucee bug, but are there any suggestions other than, simply don’t use the RAM disk?